2024-05-09T07:02:58.748 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T07:02:58.749 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_teuthology@teuthology --verbose --archive /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588 --description fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} --name teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/orig.config.yaml 2024-05-09T07:02:58.797 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T07:02:58.902 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588 branch: squid description: fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} email: ceph-qa@ceph.com first_in_suite: false job_id: '7698588' kclient-workload: sequential: - kclient: - client.1 - workunit: clients: client.1: - suites/dbench.sh kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 seed: 2334 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 359/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: be85c202969d25af6e130678e7d73a20e9c4ca11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCFRfCdf+YXts8HYJzKiBPmxgnfgXq5jIO0hPvKug4BVVpOaS3i6iuv/eYPmb3Iv3Ks9Z6C6i2vi35dthzAE5sk= smithi188.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHtyB1zHxV3GumUngXv2Sdw2J5t3fzaMi62ldMJeybmnxQ5YqGDg57qhs5T3RKsF613PJ5tgLxbKkA6G0hdQvpY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - parallel: - user-workload - kclient-workload teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_21:24:04 tube: smithi user: teuthology user-workload: sequential: - ceph-fuse: - client.0 - workunit: clients: client.0: - suites/iozone.sh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T07:02:58.902 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa; will attempt to use it 2024-05-09T07:02:58.903 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa/tasks 2024-05-09T07:02:58.903 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T07:02:58.904 INFO:teuthology.task.internal:Checking packages... 2024-05-09T07:02:58.928 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-09T07:02:58.929 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T07:02:58.929 INFO:teuthology.packaging:ref: None 2024-05-09T07:02:58.929 INFO:teuthology.packaging:tag: None 2024-05-09T07:02:58.929 INFO:teuthology.packaging:branch: squid 2024-05-09T07:02:58.929 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:02:58.929 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-09T07:02:59.229 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2529.gbf77c77e 2024-05-09T07:02:59.230 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T07:02:59.237 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T07:02:59.237 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T07:02:59.244 INFO:teuthology.task.internal:Saving configuration 2024-05-09T07:02:59.261 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T07:02:59.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T07:02:59.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:57:53.986020', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCFRfCdf+YXts8HYJzKiBPmxgnfgXq5jIO0hPvKug4BVVpOaS3i6iuv/eYPmb3Iv3Ks9Z6C6i2vi35dthzAE5sk='} 2024-05-09T07:02:59.313 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi188.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 06:57:53.984585', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHtyB1zHxV3GumUngXv2Sdw2J5t3fzaMi62ldMJeybmnxQ5YqGDg57qhs5T3RKsF613PJ5tgLxbKkA6G0hdQvpY='} 2024-05-09T07:02:59.313 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T07:02:59.319 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-05-09T07:02:59.319 INFO:teuthology.task.internal:roles: ubuntu@smithi188.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2024-05-09T07:02:59.319 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T07:02:59.395 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f00a6a80040>, signals=[15]) 2024-05-09T07:02:59.395 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T07:02:59.401 INFO:teuthology.task.internal:Opening connections... 2024-05-09T07:02:59.401 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:02:59.403 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:02:59.485 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:02:59.486 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:02:59.561 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T07:02:59.568 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2024-05-09T07:02:59.586 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2024-05-09T07:02:59.587 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="CentOS Stream" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="9" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:ID="centos" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="rhel fedora" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="9" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el9" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:LOGO="fedora-logo-icon" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://centos.org/" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T07:02:59.644 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T07:02:59.645 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T07:02:59.645 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2024-05-09T07:02:59.672 DEBUG:teuthology.orchestra.run.smithi188:> uname -m 2024-05-09T07:02:59.690 INFO:teuthology.orchestra.run.smithi188.stdout:x86_64 2024-05-09T07:02:59.690 DEBUG:teuthology.orchestra.run.smithi188:> cat /etc/os-release 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:NAME="CentOS Stream" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION="9" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:ID="centos" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:ID_LIKE="rhel fedora" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:VERSION_ID="9" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:PLATFORM_ID="platform:el9" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:ANSI_COLOR="0;31" 2024-05-09T07:02:59.747 INFO:teuthology.orchestra.run.smithi188.stdout:LOGO="fedora-logo-icon" 2024-05-09T07:02:59.748 INFO:teuthology.orchestra.run.smithi188.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T07:02:59.748 INFO:teuthology.orchestra.run.smithi188.stdout:HOME_URL="https://centos.org/" 2024-05-09T07:02:59.748 INFO:teuthology.orchestra.run.smithi188.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T07:02:59.748 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T07:02:59.748 INFO:teuthology.orchestra.run.smithi188.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T07:02:59.748 INFO:teuthology.lock.ops:Updating smithi188.front.sepia.ceph.com on lock server 2024-05-09T07:02:59.763 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T07:02:59.772 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T07:02:59.778 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T07:02:59.778 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2024-05-09T07:02:59.780 DEBUG:teuthology.orchestra.run.smithi188:> test '!' -e /home/ubuntu/cephtest 2024-05-09T07:02:59.802 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T07:02:59.809 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T07:02:59.809 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2024-05-09T07:02:59.838 DEBUG:teuthology.orchestra.run.smithi188:> test -z $(ls -A /var/lib/ceph) 2024-05-09T07:03:00.067 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T07:03:00.116 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T07:03:00.141 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T07:03:00.141 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}} 2024-05-09T07:03:00.141 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2024-05-09T07:03:00.141 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T07:03:00.142 DEBUG:teuthology.orchestra.run.smithi188:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T07:03:00.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:03:00.159 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-05-09T07:03:00.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:03:00.160 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-05-09T07:03:00.216 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-437.el9.x86_64 2024-05-09T07:03:00.217 INFO:teuthology.task.kernel:Running kernel on smithi160: 5.14.0-437.el9.x86_64 2024-05-09T07:03:00.217 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum install -y kernel 2024-05-09T07:03:00.218 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-437.el9.x86_64 2024-05-09T07:03:00.218 INFO:teuthology.task.kernel:Running kernel on smithi188: 5.14.0-437.el9.x86_64 2024-05-09T07:03:00.218 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum install -y kernel 2024-05-09T07:03:02.724 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.1 MB 00:00 2024-05-09T07:03:03.621 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-09T07:03:04.566 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-09T07:03:05.330 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-09T07:03:08.591 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-09T07:03:09.497 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-05-09T07:03:10.242 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 16 kB 00:00 2024-05-09T07:03:11.065 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-05-09T07:03:11.085 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 30 MB/s | 21 MB 00:00 2024-05-09T07:03:16.293 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-05-09T07:03:18.375 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T07:03:18.375 INFO:teuthology.orchestra.run.smithi188.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T07:03:18.470 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout: Package Architecture Version Repository Size 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T07:03:18.483 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout:================================================================================ 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout:Install 4 Packages 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 78 M 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout:Installed size: 123 M 2024-05-09T07:03:18.484 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-09T07:03:19.411 INFO:teuthology.orchestra.run.smithi188.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 588 kB/s | 484 kB 00:00 2024-05-09T07:03:30.001 INFO:teuthology.orchestra.run.smithi188.stdout:(2/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 3.0 MB/s | 34 MB 00:11 2024-05-09T07:03:31.668 INFO:teuthology.orchestra.run.smithi188.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 2.3 MB/s | 28 MB 00:12 2024-05-09T07:03:34.155 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 943 kB/s | 21 MB 00:22 2024-05-09T07:03:34.884 INFO:teuthology.orchestra.run.smithi188.stdout:(4/4): kernel-core-5.14.0-444.el9.x86_64.rpm 969 kB/s | 15 MB 00:16 2024-05-09T07:03:34.884 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-09T07:03:34.884 INFO:teuthology.orchestra.run.smithi188.stdout:Total 4.7 MB/s | 78 MB 00:16 2024-05-09T07:03:35.561 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-09T07:03:35.904 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-09T07:03:35.904 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-09T07:03:36.814 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-09T07:03:36.814 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-09T07:03:39.211 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-09T07:03:39.364 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-05-09T07:03:39.892 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T07:03:39.977 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T07:03:41.487 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T07:03:41.488 INFO:teuthology.orchestra.run.smithi160.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T07:03:41.575 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-09T07:03:41.590 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-09T07:03:41.590 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2024-05-09T07:03:41.590 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-09T07:03:41.590 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-09T07:03:41.591 INFO:teuthology.orchestra.run.smithi160.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T07:03:41.591 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-09T07:03:41.591 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T07:03:41.591 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T07:03:41.591 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T07:03:41.591 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:03:41.592 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-09T07:03:41.592 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2024-05-09T07:03:41.592 INFO:teuthology.orchestra.run.smithi160.stdout:Install 4 Packages 2024-05-09T07:03:41.592 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:03:41.593 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 78 M 2024-05-09T07:03:41.593 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 123 M 2024-05-09T07:03:41.593 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-09T07:03:42.449 INFO:teuthology.orchestra.run.smithi160.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 660 kB/s | 484 kB 00:00 2024-05-09T07:03:42.687 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T07:03:42.833 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T07:03:47.332 INFO:teuthology.orchestra.run.smithi160.stdout:(2/4): kernel-modules-core-5.14.0-444.el9.x86_6 5.7 MB/s | 28 MB 00:04 2024-05-09T07:03:47.537 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T07:03:47.585 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:03:52.326 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:03:57.622 INFO:teuthology.orchestra.run.smithi160.stdout:(3/4): kernel-core-5.14.0-444.el9.x86_64.rpm 993 kB/s | 15 MB 00:15 2024-05-09T07:04:00.655 INFO:teuthology.orchestra.run.smithi160.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 1.8 MB/s | 34 MB 00:18 2024-05-09T07:04:00.656 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-09T07:04:00.656 INFO:teuthology.orchestra.run.smithi160.stdout:Total 4.1 MB/s | 78 MB 00:19 2024-05-09T07:04:01.334 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-09T07:04:01.693 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-09T07:04:01.694 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-09T07:04:02.609 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-09T07:04:02.610 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-09T07:04:05.013 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-09T07:04:05.630 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T07:04:05.714 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T07:04:08.407 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T07:04:08.532 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T07:04:13.284 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T07:04:13.326 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:18.099 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:21.699 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:21.738 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:22.132 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:22.132 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T07:04:22.132 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T07:04:22.132 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T07:04:22.968 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:22.968 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:04:22.968 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-09T07:04:22.968 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T07:04:22.968 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T07:04:22.969 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T07:04:22.969 INFO:teuthology.orchestra.run.smithi188.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T07:04:22.969 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:04:22.969 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-09T07:04:23.331 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T07:04:23.401 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T07:04:23.401 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T07:04:23.401 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T07:04:23.401 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T07:04:23.402 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T07:04:23.402 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2024-05-09T07:04:23.402 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi188.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T07:04:23.402 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T07:04:23.453 INFO:teuthology.orchestra.run.smithi188.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T07:04:23.453 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T07:04:23.453 DEBUG:teuthology.orchestra.run.smithi188:> sudo rpm -qi grub2-tools 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Name : grub2-tools 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Epoch : 1 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Version : 2.06 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Release : 68.el9 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Architecture: x86_64 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Group : Unspecified 2024-05-09T07:04:23.487 INFO:teuthology.orchestra.run.smithi188.stdout:Size : 8264254 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:License : GPLv3+ 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:Packager : builder@centos.org 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:Vendor : CentOS 2024-05-09T07:04:23.488 INFO:teuthology.orchestra.run.smithi188.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:Summary : Support tools for GRUB. 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:Description : 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:hardware devices. 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:04:23.489 INFO:teuthology.orchestra.run.smithi188.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T07:04:23.490 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T07:04:23.490 INFO:teuthology.task.kernel:Updating grub on smithi188 to boot 5.14.0-444.el9.x86_64 2024-05-09T07:04:23.490 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T07:04:24.076 INFO:teuthology.orchestra.run.smithi188.stderr:Generating grub configuration file ... 2024-05-09T07:04:25.486 INFO:teuthology.orchestra.run.smithi188.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T07:04:25.516 INFO:teuthology.orchestra.run.smithi188.stderr:done 2024-05-09T07:04:25.518 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-05-09T07:04:25.534 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.Urg9F8nJIy 2024-05-09T07:04:25.534 DEBUG:teuthology.orchestra.run.smithi188:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Urg9F8nJIy 2024-05-09T07:04:25.601 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.Urg9F8nJIy 2024-05-09T07:04:25.740 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.Urg9F8nJIy is 6KB 2024-05-09T07:04:25.789 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.Urg9F8nJIy 2024-05-09T07:04:25.803 DEBUG:teuthology.orchestra.run.smithi188:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T07:04:25.871 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T07:04:25.871 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T07:04:25.871 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T07:04:25.871 INFO:teuthology.orchestra.run.smithi188.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T07:04:25.872 DEBUG:teuthology.orchestra.run.smithi188:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T07:04:26.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo shutdown -r now 2024-05-09T07:04:47.220 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:47.257 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:47.607 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:47.607 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T07:04:47.607 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T07:04:47.607 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T07:04:48.403 INFO:teuthology.orchestra.run.smithi160.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T07:04:48.404 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:04:48.404 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-09T07:04:48.762 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T07:04:48.838 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T07:04:48.838 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T07:04:48.839 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T07:04:48.839 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T07:04:48.839 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T07:04:48.839 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T07:04:48.839 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi160.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T07:04:48.839 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T07:04:48.888 INFO:teuthology.orchestra.run.smithi160.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T07:04:48.889 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T07:04:48.889 DEBUG:teuthology.orchestra.run.smithi160:> sudo rpm -qi grub2-tools 2024-05-09T07:04:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Name : grub2-tools 2024-05-09T07:04:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Epoch : 1 2024-05-09T07:04:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Version : 2.06 2024-05-09T07:04:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Release : 68.el9 2024-05-09T07:04:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Architecture: x86_64 2024-05-09T07:04:48.920 INFO:teuthology.orchestra.run.smithi160.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Group : Unspecified 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Size : 8264254 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:License : GPLv3+ 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Packager : builder@centos.org 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Vendor : CentOS 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Summary : Support tools for GRUB. 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:Description : 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T07:04:48.921 INFO:teuthology.orchestra.run.smithi160.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T07:04:48.922 INFO:teuthology.orchestra.run.smithi160.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T07:04:48.922 INFO:teuthology.orchestra.run.smithi160.stdout:hardware devices. 2024-05-09T07:04:48.922 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:04:48.922 INFO:teuthology.orchestra.run.smithi160.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T07:04:48.923 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T07:04:48.923 INFO:teuthology.task.kernel:Updating grub on smithi160 to boot 5.14.0-444.el9.x86_64 2024-05-09T07:04:48.923 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T07:04:49.519 INFO:teuthology.orchestra.run.smithi160.stderr:Generating grub configuration file ... 2024-05-09T07:04:50.952 INFO:teuthology.orchestra.run.smithi160.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T07:04:50.985 INFO:teuthology.orchestra.run.smithi160.stderr:done 2024-05-09T07:04:50.988 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-05-09T07:04:51.005 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.vZmT5bGUwg 2024-05-09T07:04:51.005 DEBUG:teuthology.orchestra.run.smithi160:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.vZmT5bGUwg 2024-05-09T07:04:51.077 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.vZmT5bGUwg 2024-05-09T07:04:51.207 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.vZmT5bGUwg is 6KB 2024-05-09T07:04:51.256 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.vZmT5bGUwg 2024-05-09T07:04:51.270 DEBUG:teuthology.orchestra.run.smithi160:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T07:04:51.333 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T07:04:51.333 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T07:04:51.333 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T07:04:51.334 INFO:teuthology.orchestra.run.smithi160.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T07:04:51.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T07:04:51.488 DEBUG:teuthology.orchestra.run.smithi160:> sudo shutdown -r now 2024-05-09T07:04:56.034 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-05-09T07:04:56.035 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T07:04:56.036 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:05:21.515 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T07:05:21.515 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:05:21.516 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:05:56.003 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-05-09T07:05:56.039 DEBUG:teuthology.orchestra.remote:timed out 2024-05-09T07:06:05.010 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:06:05.011 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:06:05.039 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T07:06:05.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:06:06.634 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-09T07:06:07.072 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T07:06:07.072 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.14.0-444.el9.x86_64"... 2024-05-09T07:06:07.072 DEBUG:teuthology.orchestra.run.smithi188:> uname -r 2024-05-09T07:06:07.088 INFO:teuthology.orchestra.run.smithi188.stdout:5.14.0-444.el9.x86_64 2024-05-09T07:06:07.088 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T07:06:07.088 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T07:06:07.089 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T07:06:08.090 DEBUG:teuthology.parallel:result is None 2024-05-09T07:06:23.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-05-09T07:06:35.392 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:06:35.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:06:38.462 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.160 2024-05-09T07:06:53.464 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:06:53.465 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:06:53.887 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-09T07:06:54.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:06:54.274 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T07:06:54.274 DEBUG:teuthology.orchestra.run.smithi160:> uname -r 2024-05-09T07:06:54.292 INFO:teuthology.orchestra.run.smithi160.stdout:5.14.0-444.el9.x86_64 2024-05-09T07:06:54.292 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T07:06:54.292 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T07:06:54.292 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T07:06:55.293 DEBUG:teuthology.parallel:result is None 2024-05-09T07:06:55.293 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T07:06:55.301 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T07:06:55.301 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T07:06:55.304 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T07:06:55.325 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T07:06:55.331 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T07:06:55.341 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T07:06:55.341 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T07:06:55.362 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T07:06:55.691 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T07:06:55.698 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T07:06:55.698 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T07:06:55.700 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T07:06:55.757 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:06:55.800 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:06:55.807 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:06:55.831 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T07:06:55.833 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T07:06:55.841 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T07:06:55.841 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T07:06:55.844 DEBUG:teuthology.orchestra.run.smithi188:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T07:06:55.872 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T07:06:55.881 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T07:06:55.881 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T07:06:55.914 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T07:06:55.932 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:06:56.016 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:06:56.063 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:06:56.147 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:06:56.189 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:06:56.189 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T07:06:56.255 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:06:56.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T07:06:56.351 DEBUG:teuthology.orchestra.run.smithi188:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:06:56.437 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T07:06:56.481 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:06:56.481 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T07:06:56.550 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2024-05-09T07:06:56.553 DEBUG:teuthology.orchestra.run.smithi188:> sudo service rsyslog restart 2024-05-09T07:06:56.592 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T07:06:56.653 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T07:06:57.107 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T07:06:57.114 INFO:teuthology.task.internal:Starting timer... 2024-05-09T07:06:57.114 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T07:06:57.124 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T07:06:57.133 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T07:06:57.133 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2024-05-09T07:06:57.241 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2024-05-09T07:06:57.243 DEBUG:teuthology.orchestra.run.smithi188:> sudo service auditd rotate 2024-05-09T07:06:57.325 INFO:teuthology.orchestra.run.smithi188.stdout:Rotating logs: 2024-05-09T07:06:57.327 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T07:06:57.327 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2024-05-09T07:06:57.349 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2024-05-09T07:06:57.349 DEBUG:teuthology.orchestra.run.smithi188:> /usr/sbin/getenforce 2024-05-09T07:06:57.373 INFO:teuthology.orchestra.run.smithi188.stdout:Permissive 2024-05-09T07:06:57.373 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi160.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi188.front.sepia.ceph.com': 'permissive'} 2024-05-09T07:06:57.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T07:06:57.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:06:57.418 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T07:06:57.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:06:57.447 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T07:06:57.447 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2024-05-09T07:06:57.486 DEBUG:teuthology.orchestra.run.smithi188:> sudo /usr/sbin/setenforce permissive 2024-05-09T07:06:57.516 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T07:06:57.526 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T07:06:57.544 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T07:06:57.546 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi160.front.sepia.ceph.com,smithi188.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T07:10:59.690 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi160.front.sepia.ceph.com'), Remote(name='ubuntu@smithi188.front.sepia.ceph.com')] 2024-05-09T07:10:59.691 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:10:59.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:10:59.769 DEBUG:teuthology.orchestra.run.smithi160:> true 2024-05-09T07:10:59.854 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2024-05-09T07:10:59.854 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T07:10:59.854 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi188.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T07:10:59.928 DEBUG:teuthology.orchestra.run.smithi188:> true 2024-05-09T07:11:00.002 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi188.front.sepia.ceph.com' 2024-05-09T07:11:00.003 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T07:11:00.014 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T07:11:00.014 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T07:11:00.015 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T07:11:00.017 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T07:11:00.017 DEBUG:teuthology.orchestra.run.smithi188:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T07:11:00.044 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T07:11:00.058 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T07:11:00.073 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T07:11:00.082 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2024-05-09T07:11:00.087 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T07:11:00.093 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2024-05-09T07:11:00.107 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T07:11:00.112 INFO:teuthology.orchestra.run.smithi188.stderr:sudo: ntpd: command not found 2024-05-09T07:11:00.121 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T07:11:00.123 INFO:teuthology.orchestra.run.smithi188.stdout:506 Cannot talk to daemon 2024-05-09T07:11:00.137 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T07:11:00.151 INFO:teuthology.orchestra.run.smithi188.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T07:11:00.177 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-09T07:11:00.181 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T07:11:00.181 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-09T07:11:00.181 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.181 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.181 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.181 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.205 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-09T07:11:00.208 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T07:11:00.208 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-09T07:11:00.208 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.208 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.208 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.209 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T07:11:00.209 INFO:teuthology.run_tasks:Running task install... 2024-05-09T07:11:00.217 DEBUG:teuthology.task.install:project ceph 2024-05-09T07:11:00.217 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-09T07:11:00.217 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-09T07:11:00.217 INFO:teuthology.task.install:Using flavor: default 2024-05-09T07:11:00.222 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T07:11:00.223 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T07:11:00.223 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T07:11:00.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:11:00.226 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-09T07:11:00.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:11:00.372 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T07:11:00.372 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T07:11:00.378 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-09T07:11:00.379 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-09T07:11:00.495 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T07:11:00.495 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:11:00.495 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T07:11:00.521 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T07:11:00.522 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T07:11:00.559 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T07:11:00.560 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:11:00.560 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T07:11:00.587 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T07:11:00.587 DEBUG:teuthology.orchestra.run.smithi188:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T07:11:00.588 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T07:11:00.653 DEBUG:teuthology.orchestra.run.smithi188:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T07:11:00.662 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T07:11:00.727 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2024-05-09T07:11:00.729 DEBUG:teuthology.orchestra.run.smithi188:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T07:11:00.731 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2024-05-09T07:11:00.794 INFO:teuthology.orchestra.run.smithi188.stdout:check_obsoletes = 1 2024-05-09T07:11:00.796 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum clean all 2024-05-09T07:11:00.965 INFO:teuthology.orchestra.run.smithi160.stdout:54 files removed 2024-05-09T07:11:00.993 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T07:11:01.030 INFO:teuthology.orchestra.run.smithi188.stdout:54 files removed 2024-05-09T07:11:01.057 DEBUG:teuthology.orchestra.run.smithi188:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T07:11:01.658 INFO:teuthology.orchestra.run.smithi188.stdout:ceph packages for x86_64 255 kB/s | 83 kB 00:00 2024-05-09T07:11:01.666 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 208 kB/s | 83 kB 00:00 2024-05-09T07:11:02.058 INFO:teuthology.orchestra.run.smithi188.stdout:ceph noarch packages 33 kB/s | 11 kB 00:00 2024-05-09T07:11:02.117 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-05-09T07:11:02.559 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-05-09T07:11:02.608 INFO:teuthology.orchestra.run.smithi188.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2024-05-09T07:11:03.883 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.1 MB 00:01 2024-05-09T07:11:04.367 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-09T07:11:08.885 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - AppStream 5.6 MB/s | 19 MB 00:03 2024-05-09T07:11:09.726 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - AppStream 4.0 MB/s | 19 MB 00:04 2024-05-09T07:11:13.094 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - CRB 9.1 MB/s | 6.2 MB 00:00 2024-05-09T07:11:13.802 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-09T07:11:15.370 INFO:teuthology.orchestra.run.smithi160.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 16 kB 00:01 2024-05-09T07:11:15.694 INFO:teuthology.orchestra.run.smithi188.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-09T07:11:16.696 INFO:teuthology.orchestra.run.smithi188.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-05-09T07:11:16.866 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-09T07:11:21.762 INFO:teuthology.orchestra.run.smithi188.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-05-09T07:11:22.072 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T07:11:24.114 INFO:teuthology.orchestra.run.smithi188.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:11:24.114 INFO:teuthology.orchestra.run.smithi188.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:11:24.125 INFO:teuthology.orchestra.run.smithi188.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T07:11:24.126 INFO:teuthology.orchestra.run.smithi188.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T07:11:24.127 INFO:teuthology.orchestra.run.smithi188.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T07:11:24.128 INFO:teuthology.orchestra.run.smithi188.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T07:11:24.129 INFO:teuthology.orchestra.run.smithi188.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T07:11:24.131 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T07:11:24.132 INFO:teuthology.orchestra.run.smithi188.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T07:11:24.192 INFO:teuthology.orchestra.run.smithi188.stdout:Dependencies resolved. 2024-05-09T07:11:24.214 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-09T07:11:24.214 INFO:teuthology.orchestra.run.smithi188.stdout: Package Arch Version Repository Size 2024-05-09T07:11:24.214 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-09T07:11:24.214 INFO:teuthology.orchestra.run.smithi188.stdout:Installing: 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T07:11:24.215 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T07:11:24.216 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrading: 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T07:11:24.217 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout:Installing dependencies: 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T07:11:24.218 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T07:11:24.219 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T07:11:24.220 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T07:11:24.221 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T07:11:24.222 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T07:11:24.223 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T07:11:24.224 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout:Installing weak dependencies: 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:11:24.225 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction Summary 2024-05-09T07:11:24.226 INFO:teuthology.orchestra.run.smithi188.stdout:======================================================================================= 2024-05-09T07:11:24.226 INFO:teuthology.orchestra.run.smithi188.stdout:Install 148 Packages 2024-05-09T07:11:24.226 INFO:teuthology.orchestra.run.smithi188.stdout:Upgrade 8 Packages 2024-05-09T07:11:24.226 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:11:24.226 INFO:teuthology.orchestra.run.smithi188.stdout:Total download size: 225 M 2024-05-09T07:11:24.226 INFO:teuthology.orchestra.run.smithi188.stdout:Downloading Packages: 2024-05-09T07:11:24.448 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:11:24.449 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T07:11:24.460 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T07:11:24.461 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T07:11:24.461 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T07:11:24.462 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T07:11:24.464 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T07:11:24.466 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T07:11:24.467 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T07:11:24.528 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2024-05-09T07:11:24.544 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-09T07:11:24.544 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-09T07:11:24.545 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 222 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 27 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 18 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-09T07:11:24.546 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2024-05-09T07:11:24.547 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-09T07:11:24.548 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T07:11:24.549 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T07:11:24.550 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T07:11:24.551 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T07:11:24.552 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T07:11:24.553 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T07:11:24.554 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T07:11:24.555 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout:======================================================================================= 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout:Install 148 Packages 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 8 Packages 2024-05-09T07:11:24.556 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:11:24.557 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 225 M 2024-05-09T07:11:24.557 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2024-05-09T07:11:25.390 INFO:teuthology.orchestra.run.smithi160.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-09T07:11:25.630 INFO:teuthology.orchestra.run.smithi188.stdout:(1/156): ceph-19.0.0-2515.g96805dcf.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-05-09T07:11:25.707 INFO:teuthology.orchestra.run.smithi160.stdout:(2/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 9.5 MB/s | 5.4 MB 00:00 2024-05-09T07:11:25.757 INFO:teuthology.orchestra.run.smithi160.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 2.8 MB/s | 145 kB 00:00 2024-05-09T07:11:26.108 INFO:teuthology.orchestra.run.smithi160.stdout:(4/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 6.7 MB/s | 2.4 MB 00:00 2024-05-09T07:11:26.133 INFO:teuthology.orchestra.run.smithi160.stdout:(5/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.2 MB/s | 889 kB 00:00 2024-05-09T07:11:26.147 INFO:teuthology.orchestra.run.smithi188.stdout:(2/156): ceph-fuse-19.0.0-2515.g96805dcf.el9.x8 1.7 MB/s | 889 kB 00:00 2024-05-09T07:11:26.272 INFO:teuthology.orchestra.run.smithi188.stdout:(3/156): ceph-immutable-object-cache-19.0.0-251 1.1 MB/s | 145 kB 00:00 2024-05-09T07:11:26.383 INFO:teuthology.orchestra.run.smithi160.stdout:(6/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-05-09T07:11:26.448 INFO:teuthology.orchestra.run.smithi188.stdout:(4/156): ceph-base-19.0.0-2515.g96805dcf.el9.x8 4.1 MB/s | 5.4 MB 00:01 2024-05-09T07:11:26.665 INFO:teuthology.orchestra.run.smithi188.stdout:(5/156): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86 6.9 MB/s | 1.5 MB 00:00 2024-05-09T07:11:27.249 INFO:teuthology.orchestra.run.smithi188.stdout:(6/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 8.0 MB/s | 4.7 MB 00:00 2024-05-09T07:11:27.349 INFO:teuthology.orchestra.run.smithi188.stdout:(7/156): ceph-mds-19.0.0-2515.g96805dcf.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-05-09T07:11:28.591 INFO:teuthology.orchestra.run.smithi188.stdout:(8/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 13 MB/s | 17 MB 00:01 2024-05-09T07:11:28.658 INFO:teuthology.orchestra.run.smithi188.stdout:(9/156): ceph-selinux-19.0.0-2515.g96805dcf.el9 378 kB/s | 25 kB 00:00 2024-05-09T07:11:29.575 INFO:teuthology.orchestra.run.smithi160.stdout:(7/156): ceph-common-19.0.0-2515.g96805dcf.el9. 4.7 MB/s | 21 MB 00:04 2024-05-09T07:11:29.733 INFO:teuthology.orchestra.run.smithi160.stdout:(8/156): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 5.0 MB/s | 17 MB 00:03 2024-05-09T07:11:29.800 INFO:teuthology.orchestra.run.smithi160.stdout:(9/156): ceph-mon-19.0.0-2515.g96805dcf.el9.x86 1.3 MB/s | 4.7 MB 00:03 2024-05-09T07:11:29.826 INFO:teuthology.orchestra.run.smithi160.stdout:(10/156): ceph-selinux-19.0.0-2515.g96805dcf.el 273 kB/s | 25 kB 00:00 2024-05-09T07:11:29.984 INFO:teuthology.orchestra.run.smithi160.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 1.4 MB/s | 222 kB 00:00 2024-05-09T07:11:30.068 INFO:teuthology.orchestra.run.smithi160.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 325 kB/s | 27 kB 00:00 2024-05-09T07:11:30.161 INFO:teuthology.orchestra.run.smithi160.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 346 kB/s | 32 kB 00:00 2024-05-09T07:11:30.393 INFO:teuthology.orchestra.run.smithi160.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 3.1 MB/s | 746 kB 00:00 2024-05-09T07:11:30.452 INFO:teuthology.orchestra.run.smithi160.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 2.7 MB/s | 163 kB 00:00 2024-05-09T07:11:30.502 INFO:teuthology.orchestra.run.smithi160.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 2.5 MB/s | 127 kB 00:00 2024-05-09T07:11:30.507 INFO:teuthology.orchestra.run.smithi188.stdout:(10/156): ceph-common-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 21 MB 00:05 2024-05-09T07:11:30.644 INFO:teuthology.orchestra.run.smithi160.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 3.5 MB/s | 500 kB 00:00 2024-05-09T07:11:30.666 INFO:teuthology.orchestra.run.smithi188.stdout:(11/156): cephfs-mirror-19.0.0-2515.g96805dcf.e 1.4 MB/s | 222 kB 00:00 2024-05-09T07:11:30.767 INFO:teuthology.orchestra.run.smithi188.stdout:(12/156): cephfs-shell-19.0.0-2515.g96805dcf.el 271 kB/s | 27 kB 00:00 2024-05-09T07:11:30.875 INFO:teuthology.orchestra.run.smithi188.stdout:(13/156): libcephfs-devel-19.0.0-2515.g96805dcf 293 kB/s | 32 kB 00:00 2024-05-09T07:11:31.167 INFO:teuthology.orchestra.run.smithi188.stdout:(14/156): libcephfs2-19.0.0-2515.g96805dcf.el9. 2.5 MB/s | 746 kB 00:00 2024-05-09T07:11:31.275 INFO:teuthology.orchestra.run.smithi188.stdout:(15/156): libcephsqlite-19.0.0-2515.g96805dcf.e 1.5 MB/s | 163 kB 00:00 2024-05-09T07:11:31.401 INFO:teuthology.orchestra.run.smithi188.stdout:(16/156): librados-devel-19.0.0-2515.g96805dcf. 1.0 MB/s | 127 kB 00:00 2024-05-09T07:11:31.593 INFO:teuthology.orchestra.run.smithi188.stdout:(17/156): libradosstriper1-19.0.0-2515.g96805dc 2.6 MB/s | 500 kB 00:00 2024-05-09T07:11:31.869 INFO:teuthology.orchestra.run.smithi160.stdout:(18/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 4.3 MB/s | 5.3 MB 00:01 2024-05-09T07:11:31.911 INFO:teuthology.orchestra.run.smithi160.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 1.1 MB/s | 45 kB 00:00 2024-05-09T07:11:32.028 INFO:teuthology.orchestra.run.smithi160.stdout:(20/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 4.2 MB/s | 10 MB 00:02 2024-05-09T07:11:32.053 INFO:teuthology.orchestra.run.smithi160.stdout:(21/156): python3-ceph-common-19.0.0-2515.g9680 951 kB/s | 135 kB 00:00 2024-05-09T07:11:32.095 INFO:teuthology.orchestra.run.smithi160.stdout:(22/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.4 MB/s | 162 kB 00:00 2024-05-09T07:11:32.121 INFO:teuthology.orchestra.run.smithi160.stdout:(23/156): python3-rados-19.0.0-2515.g96805dcf.e 4.7 MB/s | 321 kB 00:00 2024-05-09T07:11:32.160 INFO:teuthology.orchestra.run.smithi188.stdout:(18/156): ceph-test-19.0.0-2515.g96805dcf.el9.x 14 MB/s | 48 MB 00:03 2024-05-09T07:11:32.163 INFO:teuthology.orchestra.run.smithi160.stdout:(24/156): python3-rbd-19.0.0-2515.g96805dcf.el9 4.4 MB/s | 298 kB 00:00 2024-05-09T07:11:32.188 INFO:teuthology.orchestra.run.smithi160.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 1.4 MB/s | 99 kB 00:00 2024-05-09T07:11:32.218 INFO:teuthology.orchestra.run.smithi188.stdout:(19/156): python3-ceph-argparse-19.0.0-2515.g96 777 kB/s | 45 kB 00:00 2024-05-09T07:11:32.221 INFO:teuthology.orchestra.run.smithi160.stdout:(26/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 1.4 MB/s | 85 kB 00:00 2024-05-09T07:11:32.285 INFO:teuthology.orchestra.run.smithi188.stdout:(20/156): python3-ceph-common-19.0.0-2515.g9680 2.0 MB/s | 135 kB 00:00 2024-05-09T07:11:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:(27/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-05-09T07:11:32.352 INFO:teuthology.orchestra.run.smithi188.stdout:(21/156): python3-cephfs-19.0.0-2515.g96805dcf. 2.4 MB/s | 162 kB 00:00 2024-05-09T07:11:32.364 INFO:teuthology.orchestra.run.smithi160.stdout:(28/156): ceph-grafana-dashboards-19.0.0-2515.g 476 kB/s | 32 kB 00:00 2024-05-09T07:11:32.436 INFO:teuthology.orchestra.run.smithi188.stdout:(22/156): python3-rados-19.0.0-2515.g96805dcf.e 3.8 MB/s | 321 kB 00:00 2024-05-09T07:11:32.456 INFO:teuthology.orchestra.run.smithi160.stdout:(29/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.5 MB/s | 142 kB 00:00 2024-05-09T07:11:32.511 INFO:teuthology.orchestra.run.smithi188.stdout:(23/156): python3-rbd-19.0.0-2515.g96805dcf.el9 3.9 MB/s | 298 kB 00:00 2024-05-09T07:11:32.620 INFO:teuthology.orchestra.run.smithi188.stdout:(24/156): ceph-radosgw-19.0.0-2515.g96805dcf.el 1.9 MB/s | 10 MB 00:05 2024-05-09T07:11:32.639 INFO:teuthology.orchestra.run.smithi160.stdout:(30/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 6.8 MB/s | 3.1 MB 00:00 2024-05-09T07:11:32.645 INFO:teuthology.orchestra.run.smithi188.stdout:(25/156): python3-rgw-19.0.0-2515.g96805dcf.el9 742 kB/s | 99 kB 00:00 2024-05-09T07:11:32.720 INFO:teuthology.orchestra.run.smithi188.stdout:(26/156): librgw2-19.0.0-2515.g96805dcf.el9.x86 4.7 MB/s | 5.3 MB 00:01 2024-05-09T07:11:32.745 INFO:teuthology.orchestra.run.smithi188.stdout:(27/156): rbd-fuse-19.0.0-2515.g96805dcf.el9.x8 680 kB/s | 85 kB 00:00 2024-05-09T07:11:32.813 INFO:teuthology.orchestra.run.smithi188.stdout:(28/156): rbd-nbd-19.0.0-2515.g96805dcf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-05-09T07:11:32.838 INFO:teuthology.orchestra.run.smithi188.stdout:(29/156): ceph-grafana-dashboards-19.0.0-2515.g 345 kB/s | 32 kB 00:00 2024-05-09T07:11:32.905 INFO:teuthology.orchestra.run.smithi188.stdout:(30/156): ceph-mgr-cephadm-19.0.0-2515.g96805dc 1.5 MB/s | 142 kB 00:00 2024-05-09T07:11:32.955 INFO:teuthology.orchestra.run.smithi188.stdout:(31/156): rbd-mirror-19.0.0-2515.g96805dcf.el9. 10 MB/s | 3.1 MB 00:00 2024-05-09T07:11:33.039 INFO:teuthology.orchestra.run.smithi188.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 2.9 MB/s | 249 kB 00:00 2024-05-09T07:11:33.082 INFO:teuthology.orchestra.run.smithi160.stdout:(31/156): ceph-mgr-dashboard-19.0.0-2515.g96805 5.6 MB/s | 3.5 MB 00:00 2024-05-09T07:11:33.097 INFO:teuthology.orchestra.run.smithi188.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 847 kB/s | 49 kB 00:00 2024-05-09T07:11:33.149 INFO:teuthology.orchestra.run.smithi160.stdout:(32/156): ceph-mgr-modules-core-19.0.0-2515.g96 3.7 MB/s | 249 kB 00:00 2024-05-09T07:11:33.173 INFO:teuthology.orchestra.run.smithi188.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 219 kB/s | 16 kB 00:00 2024-05-09T07:11:33.207 INFO:teuthology.orchestra.run.smithi160.stdout:(33/156): ceph-mgr-rook-19.0.0-2515.g96805dcf.e 845 kB/s | 49 kB 00:00 2024-05-09T07:11:33.248 INFO:teuthology.orchestra.run.smithi188.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 3.4 MB/s | 264 kB 00:00 2024-05-09T07:11:33.257 INFO:teuthology.orchestra.run.smithi160.stdout:(34/156): ceph-prometheus-alerts-19.0.0-2515.g9 330 kB/s | 16 kB 00:00 2024-05-09T07:11:33.340 INFO:teuthology.orchestra.run.smithi188.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 8.1 MB/s | 763 kB 00:00 2024-05-09T07:11:33.366 INFO:teuthology.orchestra.run.smithi160.stdout:(35/156): ceph-volume-19.0.0-2515.g96805dcf.el9 2.4 MB/s | 264 kB 00:00 2024-05-09T07:11:33.399 INFO:teuthology.orchestra.run.smithi188.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 312 kB/s | 18 kB 00:00 2024-05-09T07:11:33.533 INFO:teuthology.orchestra.run.smithi160.stdout:(36/156): cephadm-19.0.0-2515.g96805dcf.el9.noa 4.5 MB/s | 763 kB 00:00 2024-05-09T07:11:33.549 INFO:teuthology.orchestra.run.smithi188.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-05-09T07:11:33.583 INFO:teuthology.orchestra.run.smithi160.stdout:(37/156): cephfs-top-19.0.0-2515.g96805dcf.el9. 366 kB/s | 18 kB 00:00 2024-05-09T07:11:33.642 INFO:teuthology.orchestra.run.smithi160.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-05-09T07:11:33.734 INFO:teuthology.orchestra.run.smithi160.stdout:(39/156): ceph-mgr-diskprediction-local-19.0.0- 6.8 MB/s | 7.4 MB 00:01 2024-05-09T07:11:33.791 INFO:teuthology.orchestra.run.smithi188.stdout:(39/156): ceph-mgr-dashboard-19.0.0-2515.g96805 3.7 MB/s | 3.5 MB 00:00 2024-05-09T07:11:33.801 INFO:teuthology.orchestra.run.smithi160.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-05-09T07:11:33.834 INFO:teuthology.orchestra.run.smithi160.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-05-09T07:11:33.841 INFO:teuthology.orchestra.run.smithi188.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.7 MB/s | 803 kB 00:00 2024-05-09T07:11:33.902 INFO:teuthology.orchestra.run.smithi160.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-05-09T07:11:33.968 INFO:teuthology.orchestra.run.smithi160.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-05-09T07:11:34.002 INFO:teuthology.orchestra.run.smithi160.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-09T07:11:34.008 INFO:teuthology.orchestra.run.smithi188.stdout:(41/156): ceph-mgr-diskprediction-local-19.0.0- 6.7 MB/s | 7.4 MB 00:01 2024-05-09T07:11:34.033 INFO:teuthology.orchestra.run.smithi188.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-09T07:11:34.059 INFO:teuthology.orchestra.run.smithi188.stdout:(43/156): libquadmath-11.4.1-3.el9.x86_64.rpm 731 kB/s | 195 kB 00:00 2024-05-09T07:11:34.060 INFO:teuthology.orchestra.run.smithi160.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-09T07:11:34.094 INFO:teuthology.orchestra.run.smithi160.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-09T07:11:34.117 INFO:teuthology.orchestra.run.smithi188.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-09T07:11:34.161 INFO:teuthology.orchestra.run.smithi160.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-05-09T07:11:34.176 INFO:teuthology.orchestra.run.smithi188.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-05-09T07:11:34.201 INFO:teuthology.orchestra.run.smithi188.stdout:(46/156): python3-ply-3.11-14.el9.noarch.rpm 636 kB/s | 106 kB 00:00 2024-05-09T07:11:34.260 INFO:teuthology.orchestra.run.smithi188.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-09T07:11:34.353 INFO:teuthology.orchestra.run.smithi160.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 2.6 MB/s | 689 kB 00:00 2024-05-09T07:11:34.394 INFO:teuthology.orchestra.run.smithi188.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 3.2 MB/s | 1.2 MB 00:00 2024-05-09T07:11:34.419 INFO:teuthology.orchestra.run.smithi188.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-05-09T07:11:34.420 INFO:teuthology.orchestra.run.smithi160.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-09T07:11:34.452 INFO:teuthology.orchestra.run.smithi188.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 648 kB/s | 21 kB 00:00 2024-05-09T07:11:34.478 INFO:teuthology.orchestra.run.smithi160.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-05-09T07:11:34.504 INFO:teuthology.orchestra.run.smithi160.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 257 kB/s | 21 kB 00:00 2024-05-09T07:11:34.511 INFO:teuthology.orchestra.run.smithi188.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-05-09T07:11:34.545 INFO:teuthology.orchestra.run.smithi188.stdout:(52/156): boost-program-options-1.75.0-8.el9.x8 709 kB/s | 107 kB 00:00 2024-05-09T07:11:34.554 INFO:teuthology.orchestra.run.smithi160.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 657 kB/s | 33 kB 00:00 2024-05-09T07:11:34.570 INFO:teuthology.orchestra.run.smithi188.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-05-09T07:11:34.579 INFO:teuthology.orchestra.run.smithi160.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-05-09T07:11:34.612 INFO:teuthology.orchestra.run.smithi188.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-05-09T07:11:34.654 INFO:teuthology.orchestra.run.smithi188.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-05-09T07:11:34.688 INFO:teuthology.orchestra.run.smithi160.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-05-09T07:11:34.713 INFO:teuthology.orchestra.run.smithi188.stdout:(56/156): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-05-09T07:11:34.738 INFO:teuthology.orchestra.run.smithi188.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-05-09T07:11:34.738 INFO:teuthology.orchestra.run.smithi160.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-05-09T07:11:34.763 INFO:teuthology.orchestra.run.smithi188.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 904 kB/s | 45 kB 00:00 2024-05-09T07:11:34.822 INFO:teuthology.orchestra.run.smithi188.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-05-09T07:11:34.847 INFO:teuthology.orchestra.run.smithi188.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.9 MB/s | 662 kB 00:00 2024-05-09T07:11:34.855 INFO:teuthology.orchestra.run.smithi160.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-05-09T07:11:34.872 INFO:teuthology.orchestra.run.smithi188.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-05-09T07:11:34.898 INFO:teuthology.orchestra.run.smithi188.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 940 kB/s | 47 kB 00:00 2024-05-09T07:11:34.914 INFO:teuthology.orchestra.run.smithi160.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-05-09T07:11:34.923 INFO:teuthology.orchestra.run.smithi188.stdout:(63/156): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-05-09T07:11:34.949 INFO:teuthology.orchestra.run.smithi188.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-05-09T07:11:34.974 INFO:teuthology.orchestra.run.smithi188.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-05-09T07:11:34.999 INFO:teuthology.orchestra.run.smithi188.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-09T07:11:35.091 INFO:teuthology.orchestra.run.smithi188.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-09T07:11:35.106 INFO:teuthology.orchestra.run.smithi160.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.4 MB/s | 662 kB 00:00 2024-05-09T07:11:35.148 INFO:teuthology.orchestra.run.smithi160.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.0 MB/s | 3.0 MB 00:00 2024-05-09T07:11:35.158 INFO:teuthology.orchestra.run.smithi188.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-05-09T07:11:35.198 INFO:teuthology.orchestra.run.smithi160.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 509 kB/s | 25 kB 00:00 2024-05-09T07:11:35.200 INFO:teuthology.orchestra.run.smithi188.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 8.2 MB/s | 911 kB 00:00 2024-05-09T07:11:35.225 INFO:teuthology.orchestra.run.smithi188.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-05-09T07:11:35.248 INFO:teuthology.orchestra.run.smithi160.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-05-09T07:11:35.274 INFO:teuthology.orchestra.run.smithi160.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-05-09T07:11:35.300 INFO:teuthology.orchestra.run.smithi188.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 13 MB/s | 1.0 MB 00:00 2024-05-09T07:11:35.332 INFO:teuthology.orchestra.run.smithi160.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-05-09T07:11:35.358 INFO:teuthology.orchestra.run.smithi160.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-09T07:11:35.374 INFO:teuthology.orchestra.run.smithi160.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-05-09T07:11:35.384 INFO:teuthology.orchestra.run.smithi188.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 31 MB/s | 5.8 MB 00:00 2024-05-09T07:11:35.409 INFO:teuthology.orchestra.run.smithi188.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-05-09T07:11:35.476 INFO:teuthology.orchestra.run.smithi188.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-05-09T07:11:35.568 INFO:teuthology.orchestra.run.smithi188.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-05-09T07:11:35.594 INFO:teuthology.orchestra.run.smithi188.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-05-09T07:11:35.619 INFO:teuthology.orchestra.run.smithi188.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-09T07:11:35.644 INFO:teuthology.orchestra.run.smithi188.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-05-09T07:11:35.670 INFO:teuthology.orchestra.run.smithi188.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-05-09T07:11:35.686 INFO:teuthology.orchestra.run.smithi188.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 830 kB/s | 35 kB 00:00 2024-05-09T07:11:35.754 INFO:teuthology.orchestra.run.smithi188.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-05-09T07:11:35.820 INFO:teuthology.orchestra.run.smithi188.stdout:(82/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-05-09T07:11:35.887 INFO:teuthology.orchestra.run.smithi188.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 23 MB/s | 5.1 MB 00:00 2024-05-09T07:11:35.913 INFO:teuthology.orchestra.run.smithi188.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-05-09T07:11:35.938 INFO:teuthology.orchestra.run.smithi188.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 841 kB/s | 42 kB 00:00 2024-05-09T07:11:35.967 INFO:teuthology.orchestra.run.smithi188.stdout:(86/156): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 222 kB 00:00 2024-05-09T07:11:35.997 INFO:teuthology.orchestra.run.smithi188.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 269 kB 00:00 2024-05-09T07:11:36.022 INFO:teuthology.orchestra.run.smithi188.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-05-09T07:11:36.073 INFO:teuthology.orchestra.run.smithi188.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-05-09T07:11:36.098 INFO:teuthology.orchestra.run.smithi188.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-05-09T07:11:36.123 INFO:teuthology.orchestra.run.smithi188.stdout:(91/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-05-09T07:11:36.149 INFO:teuthology.orchestra.run.smithi188.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-05-09T07:11:36.174 INFO:teuthology.orchestra.run.smithi188.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 878 kB/s | 44 kB 00:00 2024-05-09T07:11:36.216 INFO:teuthology.orchestra.run.smithi188.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-05-09T07:11:36.233 INFO:teuthology.orchestra.run.smithi160.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 5.2 MB/s | 4.4 MB 00:00 2024-05-09T07:11:36.241 INFO:teuthology.orchestra.run.smithi188.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.3 MB/s | 221 kB 00:00 2024-05-09T07:11:36.267 INFO:teuthology.orchestra.run.smithi188.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-09T07:11:36.309 INFO:teuthology.orchestra.run.smithi188.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-05-09T07:11:36.334 INFO:teuthology.orchestra.run.smithi188.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-05-09T07:11:36.359 INFO:teuthology.orchestra.run.smithi188.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T07:11:36.384 INFO:teuthology.orchestra.run.smithi188.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T07:11:36.408 INFO:teuthology.orchestra.run.smithi160.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 4.5 MB/s | 4.7 MB 00:01 2024-05-09T07:11:36.418 INFO:teuthology.orchestra.run.smithi188.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-05-09T07:11:36.468 INFO:teuthology.orchestra.run.smithi188.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-05-09T07:11:36.584 INFO:teuthology.orchestra.run.smithi160.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-05-09T07:11:36.643 INFO:teuthology.orchestra.run.smithi160.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-05-09T07:11:36.652 INFO:teuthology.orchestra.run.smithi188.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-09T07:11:36.719 INFO:teuthology.orchestra.run.smithi188.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.8 MB/s | 863 kB 00:00 2024-05-09T07:11:36.748 INFO:teuthology.orchestra.run.smithi160.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 2.6 MB/s | 911 kB 00:00 2024-05-09T07:11:37.068 INFO:teuthology.orchestra.run.smithi160.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.1 MB/s | 1.0 MB 00:00 2024-05-09T07:11:37.111 INFO:teuthology.orchestra.run.smithi188.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 50 kB/s | 19 kB 00:00 2024-05-09T07:11:37.127 INFO:teuthology.orchestra.run.smithi160.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-09T07:11:37.336 INFO:teuthology.orchestra.run.smithi188.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 632 kB/s | 548 kB 00:00 2024-05-09T07:11:37.362 INFO:teuthology.orchestra.run.smithi188.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 453 kB/s | 309 kB 00:00 2024-05-09T07:11:37.446 INFO:teuthology.orchestra.run.smithi188.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-05-09T07:11:37.479 INFO:teuthology.orchestra.run.smithi188.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 415 kB/s | 49 kB 00:00 2024-05-09T07:11:37.563 INFO:teuthology.orchestra.run.smithi188.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 577 kB/s | 67 kB 00:00 2024-05-09T07:11:37.613 INFO:teuthology.orchestra.run.smithi188.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-05-09T07:11:37.702 INFO:teuthology.orchestra.run.smithi160.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 5.5 MB/s | 5.8 MB 00:01 2024-05-09T07:11:37.780 INFO:teuthology.orchestra.run.smithi188.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.8 MB/s | 839 kB 00:00 2024-05-09T07:11:37.786 INFO:teuthology.orchestra.run.smithi160.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-09T07:11:37.822 INFO:teuthology.orchestra.run.smithi188.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 2.6 MB/s | 548 kB 00:00 2024-05-09T07:11:37.869 INFO:teuthology.orchestra.run.smithi160.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-09T07:11:37.888 INFO:teuthology.orchestra.run.smithi188.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-05-09T07:11:37.920 INFO:teuthology.orchestra.run.smithi160.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 513 kB/s | 26 kB 00:00 2024-05-09T07:11:37.931 INFO:teuthology.orchestra.run.smithi188.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-05-09T07:11:37.970 INFO:teuthology.orchestra.run.smithi160.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-05-09T07:11:37.989 INFO:teuthology.orchestra.run.smithi188.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-05-09T07:11:38.028 INFO:teuthology.orchestra.run.smithi160.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-09T07:11:38.048 INFO:teuthology.orchestra.run.smithi188.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-05-09T07:11:38.079 INFO:teuthology.orchestra.run.smithi160.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 696 kB/s | 35 kB 00:00 2024-05-09T07:11:38.107 INFO:teuthology.orchestra.run.smithi188.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-05-09T07:11:38.174 INFO:teuthology.orchestra.run.smithi188.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.2 MB/s | 4.4 MB 00:01 2024-05-09T07:11:38.199 INFO:teuthology.orchestra.run.smithi188.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-09T07:11:38.216 INFO:teuthology.orchestra.run.smithi188.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 324 kB/s | 35 kB 00:00 2024-05-09T07:11:38.341 INFO:teuthology.orchestra.run.smithi188.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-05-09T07:11:38.433 INFO:teuthology.orchestra.run.smithi188.stdout:(123/156): python3-google-auth-2.29.0-1.el9.noa 794 kB/s | 206 kB 00:00 2024-05-09T07:11:38.458 INFO:teuthology.orchestra.run.smithi188.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-05-09T07:11:38.500 INFO:teuthology.orchestra.run.smithi188.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.8 MB/s | 2.0 MB 00:00 2024-05-09T07:11:38.542 INFO:teuthology.orchestra.run.smithi188.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 162 kB/s | 18 kB 00:00 2024-05-09T07:11:38.568 INFO:teuthology.orchestra.run.smithi188.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 213 kB/s | 23 kB 00:00 2024-05-09T07:11:38.601 INFO:teuthology.orchestra.run.smithi188.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-09T07:11:38.651 INFO:teuthology.orchestra.run.smithi188.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 181 kB/s | 20 kB 00:00 2024-05-09T07:11:38.710 INFO:teuthology.orchestra.run.smithi188.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 425 kB/s | 46 kB 00:00 2024-05-09T07:11:38.752 INFO:teuthology.orchestra.run.smithi188.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-05-09T07:11:38.777 INFO:teuthology.orchestra.run.smithi188.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 628 kB/s | 79 kB 00:00 2024-05-09T07:11:38.787 INFO:teuthology.orchestra.run.smithi160.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.6 MB/s | 6.0 MB 00:01 2024-05-09T07:11:38.819 INFO:teuthology.orchestra.run.smithi188.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 530 kB/s | 58 kB 00:00 2024-05-09T07:11:38.861 INFO:teuthology.orchestra.run.smithi188.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-05-09T07:11:38.871 INFO:teuthology.orchestra.run.smithi160.stdout:(81/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.4 MB/s | 5.1 MB 00:00 2024-05-09T07:11:38.886 INFO:teuthology.orchestra.run.smithi188.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 151 kB/s | 16 kB 00:00 2024-05-09T07:11:38.913 INFO:teuthology.orchestra.run.smithi160.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-05-09T07:11:38.928 INFO:teuthology.orchestra.run.smithi188.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 825 kB/s | 90 kB 00:00 2024-05-09T07:11:38.938 INFO:teuthology.orchestra.run.smithi160.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.3 MB/s | 222 kB 00:00 2024-05-09T07:11:38.955 INFO:teuthology.orchestra.run.smithi160.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-05-09T07:11:38.970 INFO:teuthology.orchestra.run.smithi188.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 259 kB/s | 28 kB 00:00 2024-05-09T07:11:38.989 INFO:teuthology.orchestra.run.smithi160.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-05-09T07:11:38.996 INFO:teuthology.orchestra.run.smithi188.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 282 kB/s | 31 kB 00:00 2024-05-09T07:11:39.038 INFO:teuthology.orchestra.run.smithi188.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-05-09T07:11:39.047 INFO:teuthology.orchestra.run.smithi160.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-05-09T07:11:39.073 INFO:teuthology.orchestra.run.smithi160.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-05-09T07:11:39.079 INFO:teuthology.orchestra.run.smithi188.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 543 kB/s | 59 kB 00:00 2024-05-09T07:11:39.105 INFO:teuthology.orchestra.run.smithi188.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 328 kB/s | 36 kB 00:00 2024-05-09T07:11:39.123 INFO:teuthology.orchestra.run.smithi160.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-09T07:11:39.148 INFO:teuthology.orchestra.run.smithi160.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-09T07:11:39.163 INFO:teuthology.orchestra.run.smithi188.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 406 kB/s | 51 kB 00:00 2024-05-09T07:11:39.174 INFO:teuthology.orchestra.run.smithi160.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-05-09T07:11:39.189 INFO:teuthology.orchestra.run.smithi188.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-05-09T07:11:39.214 INFO:teuthology.orchestra.run.smithi188.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 822 kB/s | 90 kB 00:00 2024-05-09T07:11:39.224 INFO:teuthology.orchestra.run.smithi160.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 836 kB/s | 42 kB 00:00 2024-05-09T07:11:39.274 INFO:teuthology.orchestra.run.smithi160.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 884 kB/s | 44 kB 00:00 2024-05-09T07:11:39.281 INFO:teuthology.orchestra.run.smithi188.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-05-09T07:11:39.306 INFO:teuthology.orchestra.run.smithi188.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-05-09T07:11:39.324 INFO:teuthology.orchestra.run.smithi188.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-09T07:11:39.358 INFO:teuthology.orchestra.run.smithi160.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T07:11:39.408 INFO:teuthology.orchestra.run.smithi160.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 715 kB/s | 36 kB 00:00 2024-05-09T07:11:39.441 INFO:teuthology.orchestra.run.smithi188.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-05-09T07:11:39.467 INFO:teuthology.orchestra.run.smithi160.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 813 kB/s | 47 kB 00:00 2024-05-09T07:11:39.584 INFO:teuthology.orchestra.run.smithi160.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.6 MB/s | 305 kB 00:00 2024-05-09T07:11:39.634 INFO:teuthology.orchestra.run.smithi160.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-05-09T07:11:39.685 INFO:teuthology.orchestra.run.smithi160.stdout:(98/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-05-09T07:11:39.735 INFO:teuthology.orchestra.run.smithi160.stdout:(99/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-05-09T07:11:39.793 INFO:teuthology.orchestra.run.smithi160.stdout:(100/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 770 kB/s | 45 kB 00:00 2024-05-09T07:11:39.844 INFO:teuthology.orchestra.run.smithi160.stdout:(101/156): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-05-09T07:11:39.924 INFO:teuthology.orchestra.run.smithi188.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 4.8 MB/s | 2.3 MB 00:00 2024-05-09T07:11:40.111 INFO:teuthology.orchestra.run.smithi160.stdout:(102/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.2 MB/s | 863 kB 00:00 2024-05-09T07:11:40.149 INFO:teuthology.orchestra.run.smithi188.stdout:(150/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 3.8 MB/s | 3.1 MB 00:00 2024-05-09T07:11:40.199 INFO:teuthology.orchestra.run.smithi188.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 840 kB/s | 42 kB 00:00 2024-05-09T07:11:40.283 INFO:teuthology.orchestra.run.smithi188.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.9 MB/s | 248 kB 00:00 2024-05-09T07:11:40.409 INFO:teuthology.orchestra.run.smithi188.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 3.8 MB/s | 1.8 MB 00:00 2024-05-09T07:11:40.526 INFO:teuthology.orchestra.run.smithi188.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 1.3 MB 00:00 2024-05-09T07:11:40.753 INFO:teuthology.orchestra.run.smithi160.stdout:(103/156): ceph-test-19.0.0-2515.g96805dcf.el9. 4.4 MB/s | 48 MB 00:10 2024-05-09T07:11:40.834 INFO:teuthology.orchestra.run.smithi188.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.6 MB/s | 2.4 MB 00:00 2024-05-09T07:11:40.878 INFO:teuthology.orchestra.run.smithi160.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 715 kB/s | 548 kB 00:00 2024-05-09T07:11:40.937 INFO:teuthology.orchestra.run.smithi160.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-05-09T07:11:41.001 INFO:teuthology.orchestra.run.smithi188.stdout:(156/156): librados2-19.0.0-2515.g96805dcf.el9. 2.0 MB/s | 3.4 MB 00:01 2024-05-09T07:11:41.007 INFO:teuthology.orchestra.run.smithi188.stdout:-------------------------------------------------------------------------------- 2024-05-09T07:11:41.007 INFO:teuthology.orchestra.run.smithi188.stdout:Total 13 MB/s | 225 MB 00:16 2024-05-09T07:11:41.070 INFO:teuthology.orchestra.run.smithi160.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 975 kB/s | 309 kB 00:00 2024-05-09T07:11:41.129 INFO:teuthology.orchestra.run.smithi160.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-05-09T07:11:41.296 INFO:teuthology.orchestra.run.smithi160.stdout:(108/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.6 MB/s | 16 MB 00:02 2024-05-09T07:11:41.438 INFO:teuthology.orchestra.run.smithi160.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 157 kB/s | 49 kB 00:00 2024-05-09T07:11:41.554 INFO:teuthology.orchestra.run.smithi160.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.7 MB/s | 151 kB 00:00 2024-05-09T07:11:41.621 INFO:teuthology.orchestra.run.smithi160.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 207 kB/s | 67 kB 00:00 2024-05-09T07:11:41.955 INFO:teuthology.orchestra.run.smithi160.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-05-09T07:11:41.980 INFO:teuthology.orchestra.run.smithi160.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-05-09T07:11:42.014 INFO:teuthology.orchestra.run.smithi160.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 1.4 MB/s | 548 kB 00:00 2024-05-09T07:11:42.039 INFO:teuthology.orchestra.run.smithi160.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 518 kB/s | 43 kB 00:00 2024-05-09T07:11:42.065 INFO:teuthology.orchestra.run.smithi160.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 383 kB/s | 32 kB 00:00 2024-05-09T07:11:42.090 INFO:teuthology.orchestra.run.smithi160.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-09T07:11:42.123 INFO:teuthology.orchestra.run.smithi160.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-09T07:11:42.182 INFO:teuthology.orchestra.run.smithi160.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 606 kB/s | 35 kB 00:00 2024-05-09T07:11:42.216 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction check 2024-05-09T07:11:42.216 INFO:teuthology.orchestra.run.smithi160.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 2.3 MB/s | 358 kB 00:00 2024-05-09T07:11:42.241 INFO:teuthology.orchestra.run.smithi160.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-05-09T07:11:42.266 INFO:teuthology.orchestra.run.smithi160.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-05-09T07:11:42.308 INFO:teuthology.orchestra.run.smithi160.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.1 MB/s | 144 kB 00:00 2024-05-09T07:11:42.334 INFO:teuthology.orchestra.run.smithi160.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 159 kB/s | 11 kB 00:00 2024-05-09T07:11:42.367 INFO:teuthology.orchestra.run.smithi160.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 302 kB/s | 18 kB 00:00 2024-05-09T07:11:42.368 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction check succeeded. 2024-05-09T07:11:42.368 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction test 2024-05-09T07:11:42.392 INFO:teuthology.orchestra.run.smithi160.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-05-09T07:11:42.426 INFO:teuthology.orchestra.run.smithi160.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 333 kB/s | 19 kB 00:00 2024-05-09T07:11:42.451 INFO:teuthology.orchestra.run.smithi160.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 336 kB/s | 20 kB 00:00 2024-05-09T07:11:42.510 INFO:teuthology.orchestra.run.smithi160.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 794 kB/s | 46 kB 00:00 2024-05-09T07:11:42.569 INFO:teuthology.orchestra.run.smithi160.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-05-09T07:11:42.635 INFO:teuthology.orchestra.run.smithi160.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 4.9 MB/s | 1.0 MB 00:00 2024-05-09T07:11:42.661 INFO:teuthology.orchestra.run.smithi160.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 629 kB/s | 58 kB 00:00 2024-05-09T07:11:42.711 INFO:teuthology.orchestra.run.smithi160.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T07:11:42.736 INFO:teuthology.orchestra.run.smithi160.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-05-09T07:11:42.778 INFO:teuthology.orchestra.run.smithi160.stdout:(135/156): python3-grpcio-1.46.7-10.el9.x86_64. 3.6 MB/s | 2.0 MB 00:00 2024-05-09T07:11:42.803 INFO:teuthology.orchestra.run.smithi160.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-05-09T07:11:42.829 INFO:teuthology.orchestra.run.smithi160.stdout:(137/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 767 kB/s | 90 kB 00:00 2024-05-09T07:11:42.846 INFO:teuthology.orchestra.run.smithi160.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-05-09T07:11:42.880 INFO:teuthology.orchestra.run.smithi160.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-05-09T07:11:42.905 INFO:teuthology.orchestra.run.smithi160.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T07:11:42.921 INFO:teuthology.orchestra.run.smithi160.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 477 kB/s | 36 kB 00:00 2024-05-09T07:11:42.947 INFO:teuthology.orchestra.run.smithi160.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 763 kB/s | 51 kB 00:00 2024-05-09T07:11:42.980 INFO:teuthology.orchestra.run.smithi160.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-09T07:11:43.006 INFO:teuthology.orchestra.run.smithi160.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-05-09T07:11:43.048 INFO:teuthology.orchestra.run.smithi160.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.1 MB/s | 427 kB 00:00 2024-05-09T07:11:43.073 INFO:teuthology.orchestra.run.smithi160.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-05-09T07:11:43.098 INFO:teuthology.orchestra.run.smithi160.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-05-09T07:11:43.265 INFO:teuthology.orchestra.run.smithi160.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 8.2 MB/s | 1.6 MB 00:00 2024-05-09T07:11:43.407 INFO:teuthology.orchestra.run.smithi160.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 16 MB/s | 2.3 MB 00:00 2024-05-09T07:11:43.566 INFO:teuthology.orchestra.run.smithi160.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 12 MB/s | 1.8 MB 00:00 2024-05-09T07:11:43.624 INFO:teuthology.orchestra.run.smithi160.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 719 kB/s | 42 kB 00:00 2024-05-09T07:11:43.741 INFO:teuthology.orchestra.run.smithi160.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.1 MB/s | 248 kB 00:00 2024-05-09T07:11:43.875 INFO:teuthology.orchestra.run.smithi160.stdout:(153/156): openssl-3.2.1-1.el9.x86_64.rpm 10 MB/s | 1.3 MB 00:00 2024-05-09T07:11:44.075 INFO:teuthology.orchestra.run.smithi160.stdout:(154/156): librados2-19.0.0-2515.g96805dcf.el9. 3.4 MB/s | 3.4 MB 00:01 2024-05-09T07:11:44.117 INFO:teuthology.orchestra.run.smithi160.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-05-09T07:11:44.334 INFO:teuthology.orchestra.run.smithi160.stdout:(156/156): librbd1-19.0.0-2515.g96805dcf.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-05-09T07:11:44.340 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2024-05-09T07:11:44.340 INFO:teuthology.orchestra.run.smithi160.stdout:Total 11 MB/s | 225 MB 00:19 2024-05-09T07:11:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:Transaction test succeeded. 2024-05-09T07:11:44.788 INFO:teuthology.orchestra.run.smithi188.stdout:Running transaction 2024-05-09T07:11:45.542 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2024-05-09T07:11:45.692 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2024-05-09T07:11:45.692 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2024-05-09T07:11:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2024-05-09T07:11:47.878 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2024-05-09T07:11:50.960 INFO:teuthology.orchestra.run.smithi188.stdout: Preparing : 1/1 2024-05-09T07:11:51.046 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T07:11:51.085 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T07:11:51.156 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T07:11:51.189 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T07:11:51.214 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T07:11:51.272 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T07:11:51.606 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T07:11:51.724 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T07:11:51.855 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T07:11:51.875 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T07:11:51.967 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T07:11:51.996 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T07:11:52.178 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T07:11:52.227 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T07:11:52.254 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T07:11:52.367 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T07:11:52.389 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T07:11:52.443 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T07:11:52.465 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T07:11:52.486 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T07:11:52.683 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T07:11:52.873 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T07:11:53.027 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T07:11:53.090 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T07:11:53.214 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T07:11:53.292 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T07:11:53.501 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T07:11:53.618 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T07:11:53.710 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T07:11:53.740 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T07:11:53.775 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T07:11:53.839 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T07:11:53.875 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T07:11:53.917 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T07:11:53.950 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T07:11:53.977 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T07:11:54.046 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T07:11:54.099 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T07:11:54.139 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T07:11:54.174 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T07:11:54.215 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2024-05-09T07:11:54.249 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T07:11:54.284 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T07:11:54.289 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T07:11:54.329 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T07:11:54.330 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T07:11:54.357 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T07:11:54.390 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T07:11:54.426 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T07:11:54.430 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T07:11:54.456 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T07:11:54.472 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T07:11:54.506 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T07:11:54.546 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T07:11:54.677 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T07:11:54.732 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T07:11:54.847 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T07:11:54.941 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T07:11:55.064 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T07:11:55.091 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T07:11:55.184 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T07:11:55.212 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T07:11:55.247 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T07:11:55.328 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T07:11:55.366 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T07:11:55.383 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T07:11:55.409 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T07:11:55.447 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T07:11:55.447 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T07:11:55.479 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T07:11:55.488 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T07:11:55.583 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T07:11:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T07:11:55.636 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T07:11:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 14/164 2024-05-09T07:11:55.691 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T07:11:55.706 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T07:11:55.729 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T07:11:55.745 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T07:11:55.810 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T07:11:55.842 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T07:11:55.878 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T07:11:55.925 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T07:11:55.925 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T07:11:55.961 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T07:11:56.004 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T07:11:56.046 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T07:11:56.092 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T07:11:56.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T07:11:56.219 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T07:11:56.236 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T07:11:56.249 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T07:11:56.290 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T07:11:56.296 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T07:11:56.384 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T07:11:56.420 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T07:11:56.425 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T07:11:56.481 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T07:11:56.505 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T07:11:56.677 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T07:11:56.709 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T07:11:56.774 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T07:11:56.845 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T07:11:56.867 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T07:11:56.951 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T07:11:56.991 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T07:11:57.004 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-05-09T07:11:57.031 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T07:11:57.046 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-05-09T07:11:57.059 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T07:11:57.093 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T07:11:57.120 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-05-09T07:11:57.132 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T07:11:57.164 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T07:11:57.170 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-05-09T07:11:57.194 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T07:11:57.214 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/164 2024-05-09T07:11:57.228 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T07:11:57.246 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 26/164 2024-05-09T07:11:57.262 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T07:11:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/164 2024-05-09T07:11:57.303 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T07:11:57.338 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T07:11:57.343 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/164 2024-05-09T07:11:57.397 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 29/164 2024-05-09T07:11:57.425 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T07:11:57.427 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 30/164 2024-05-09T07:11:57.471 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 31/164 2024-05-09T07:11:57.477 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T07:11:57.508 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T07:11:57.547 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 32/164 2024-05-09T07:11:57.554 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T07:11:57.581 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 33/164 2024-05-09T07:11:57.633 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T07:11:57.652 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T07:11:57.661 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 35/164 2024-05-09T07:11:57.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T07:11:57.731 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 36/164 2024-05-09T07:11:57.776 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : luarocks-3.9.2-1.el9.noarch 37/164 2024-05-09T07:11:57.850 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 38/164 2024-05-09T07:11:57.984 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 39/164 2024-05-09T07:11:58.045 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 40/164 2024-05-09T07:11:58.435 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T07:11:58.482 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T07:11:58.515 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T07:11:58.565 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el9.noarch 41/164 2024-05-09T07:11:58.613 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T07:11:58.641 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 42/164 2024-05-09T07:11:58.664 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T07:11:58.670 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 43/164 2024-05-09T07:11:58.713 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 44/164 2024-05-09T07:11:58.751 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 45/164 2024-05-09T07:11:58.751 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T07:11:58.793 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 46/164 2024-05-09T07:11:58.801 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T07:11:58.943 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 47/164 2024-05-09T07:11:58.979 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T07:11:59.003 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 48/164 2024-05-09T07:11:59.007 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T07:11:59.058 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 49/164 2024-05-09T07:11:59.091 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T07:11:59.130 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T07:11:59.162 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T07:11:59.199 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T07:11:59.245 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T07:11:59.290 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T07:11:59.341 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T07:11:59.384 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T07:11:59.520 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T07:11:59.564 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T07:11:59.594 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T07:11:59.635 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T07:11:59.729 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T07:11:59.759 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T07:11:59.769 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T07:11:59.803 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T07:11:59.834 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T07:11:59.859 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T07:11:59.871 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T07:11:59.957 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T07:12:00.087 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T07:12:00.117 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T07:12:00.148 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T07:12:00.179 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T07:12:00.218 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T07:12:00.249 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T07:12:00.279 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T07:12:00.314 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T07:12:00.347 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T07:12:00.389 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T07:12:00.423 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T07:12:00.510 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T07:12:00.560 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T07:12:00.593 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T07:12:00.640 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T07:12:00.739 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T07:12:00.795 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T07:12:01.108 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T07:12:01.212 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T07:12:01.239 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T07:12:01.273 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T07:12:01.522 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T07:12:01.566 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T07:12:01.591 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T07:12:01.600 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T07:12:01.636 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T07:12:01.698 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T07:12:01.726 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T07:12:01.750 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T07:12:01.757 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T07:12:01.794 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T07:12:01.836 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T07:12:01.877 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T07:12:02.058 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T07:12:02.109 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T07:12:02.197 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T07:12:02.270 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T07:12:02.297 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T07:12:02.369 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T07:12:02.399 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T07:12:02.844 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T07:12:02.947 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T07:12:04.221 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T07:12:04.303 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T07:12:04.341 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T07:12:04.373 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T07:12:04.656 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T07:12:04.687 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T07:12:04.691 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T07:12:04.727 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T07:12:04.810 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 101/164 2024-05-09T07:12:04.840 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 102/164 2024-05-09T07:12:04.877 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T07:12:05.236 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T07:12:05.274 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T07:12:05.351 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T07:12:05.366 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T07:12:05.366 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T07:12:05.366 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T07:12:05.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:05.380 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T07:12:05.400 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T07:12:05.451 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 105/164 2024-05-09T07:12:05.482 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 106/164 2024-05-09T07:12:07.829 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T07:12:07.870 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T07:12:08.425 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 107/164 2024-05-09T07:12:08.465 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T07:12:08.543 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 108/164 2024-05-09T07:12:08.543 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T07:12:08.543 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T07:12:08.543 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:08.582 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /sys 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /proc 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /mnt 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /var/tmp 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /home 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /root 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout:skipping the directory /tmp 2024-05-09T07:12:17.187 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:17.350 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T07:12:17.366 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T07:12:17.366 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:17.366 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T07:12:17.366 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:18.878 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T07:12:18.904 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T07:12:18.904 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:18.904 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T07:12:18.904 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T07:12:18.904 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T07:12:18.905 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:19.137 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T07:12:19.160 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T07:12:19.160 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:19.160 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T07:12:19.160 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T07:12:19.160 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T07:12:19.160 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:19.628 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T07:12:19.652 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T07:12:19.652 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:19.652 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T07:12:19.652 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T07:12:19.652 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T07:12:19.652 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:19.758 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T07:12:19.807 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T07:12:19.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T07:12:19.880 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 109/164 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /root 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2024-05-09T07:12:19.881 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:20.055 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T07:12:20.082 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T07:12:20.099 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 110/164 2024-05-09T07:12:20.099 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:20.099 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T07:12:20.099 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:20.104 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T07:12:20.181 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T07:12:20.270 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T07:12:20.408 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T07:12:20.454 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T07:12:20.491 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T07:12:20.522 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T07:12:20.981 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T07:12:21.012 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T07:12:21.044 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T07:12:21.106 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T07:12:21.135 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T07:12:21.600 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T07:12:21.627 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 111/164 2024-05-09T07:12:21.627 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:21.627 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T07:12:21.627 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T07:12:21.627 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T07:12:21.628 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:21.859 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T07:12:21.882 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 112/164 2024-05-09T07:12:21.883 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:21.883 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T07:12:21.883 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T07:12:21.883 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T07:12:21.883 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:22.163 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T07:12:22.205 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T07:12:22.343 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T07:12:22.367 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 113/164 2024-05-09T07:12:22.367 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:22.367 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T07:12:22.367 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T07:12:22.367 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T07:12:22.367 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:22.481 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T07:12:22.530 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T07:12:22.585 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T07:12:22.780 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T07:12:22.835 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T07:12:22.913 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T07:12:23.001 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T07:12:23.140 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T07:12:23.174 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T07:12:23.184 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T07:12:23.212 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T07:12:23.227 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T07:12:23.253 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T07:12:23.313 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T07:12:23.497 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T07:12:23.523 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T07:12:23.546 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T07:12:23.546 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:23.546 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T07:12:23.546 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T07:12:23.547 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T07:12:23.547 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:23.598 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T07:12:23.610 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T07:12:23.703 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T07:12:23.708 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T07:12:23.734 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 125/164 2024-05-09T07:12:23.761 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T07:12:23.766 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 126/164 2024-05-09T07:12:23.828 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 127/164 2024-05-09T07:12:23.857 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T07:12:24.662 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T07:12:24.709 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T07:12:24.732 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T07:12:24.732 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:24.732 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T07:12:24.732 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T07:12:24.733 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T07:12:24.733 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:24.776 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T07:12:24.799 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T07:12:24.799 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:24.799 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T07:12:24.799 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:24.843 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T07:12:24.866 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T07:12:24.866 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:24.866 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T07:12:24.867 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T07:12:24.867 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T07:12:24.867 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:24.904 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 128/164 2024-05-09T07:12:24.934 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T07:12:25.177 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T07:12:25.200 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T07:12:25.201 INFO:teuthology.orchestra.run.smithi188.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:25.201 INFO:teuthology.orchestra.run.smithi188.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T07:12:25.201 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T07:12:25.201 INFO:teuthology.orchestra.run.smithi188.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T07:12:25.201 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:25.911 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 129/164 2024-05-09T07:12:25.934 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T07:12:26.039 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 130/164 2024-05-09T07:12:26.206 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 131/164 2024-05-09T07:12:26.236 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T07:12:26.260 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 132/164 2024-05-09T07:12:26.261 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:26.261 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T07:12:26.261 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T07:12:26.261 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T07:12:26.261 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:26.312 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T07:12:26.324 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 133/164 2024-05-09T07:12:26.422 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 134/164 2024-05-09T07:12:26.485 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 135/164 2024-05-09T07:12:27.398 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T07:12:27.422 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T07:12:27.446 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 137/164 2024-05-09T07:12:27.446 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:27.446 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T07:12:27.446 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T07:12:27.446 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T07:12:27.447 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:27.490 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T07:12:27.513 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805 138/164 2024-05-09T07:12:27.513 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:27.513 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T07:12:27.513 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:27.565 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T07:12:27.588 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 139/164 2024-05-09T07:12:27.589 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:27.589 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T07:12:27.589 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T07:12:27.589 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T07:12:27.589 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:27.891 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T07:12:27.915 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 140/164 2024-05-09T07:12:27.915 INFO:teuthology.orchestra.run.smithi160.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T07:12:27.915 INFO:teuthology.orchestra.run.smithi160.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T07:12:27.915 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T07:12:27.915 INFO:teuthology.orchestra.run.smithi160.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T07:12:27.915 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:29.780 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T07:12:29.822 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T07:12:29.862 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T07:12:29.902 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T07:12:29.930 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T07:12:30.120 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T07:12:30.160 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T07:12:30.190 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T07:12:30.219 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T07:12:30.274 INFO:teuthology.orchestra.run.smithi188.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T07:12:30.899 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T07:12:30.930 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T07:12:30.950 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T07:12:31.775 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T07:12:31.851 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T07:12:31.901 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T07:12:31.987 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T07:12:32.022 INFO:teuthology.orchestra.run.smithi188.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T07:12:32.047 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T07:12:32.047 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T07:12:32.131 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T07:12:32.164 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T07:12:32.164 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T07:12:32.249 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T07:12:32.274 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T07:12:32.279 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T07:12:32.279 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T07:12:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T07:12:32.467 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T07:12:32.478 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 141/164 2024-05-09T07:12:32.519 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 142/164 2024-05-09T07:12:32.551 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T07:12:32.560 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 143/164 2024-05-09T07:12:32.592 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T07:12:32.616 INFO:teuthology.orchestra.run.smithi188.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T07:12:32.619 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T07:12:32.810 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 146/164 2024-05-09T07:12:32.857 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T07:12:32.887 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 148/164 2024-05-09T07:12:32.917 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T07:12:32.954 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T07:12:33.581 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T07:12:33.618 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T07:12:33.638 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T07:12:34.485 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 152/164 2024-05-09T07:12:34.564 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T07:12:34.615 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T07:12:34.712 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T07:12:34.750 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T07:12:34.776 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T07:12:34.776 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T07:12:34.860 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T07:12:34.893 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T07:12:34.893 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T07:12:34.961 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T07:12:34.986 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T07:12:34.991 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T07:12:34.991 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T07:12:35.172 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T07:12:35.172 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T07:12:35.264 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T07:12:35.329 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T07:12:38.087 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T07:12:38.088 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T07:12:38.089 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T07:12:38.089 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T07:12:38.089 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T07:12:38.089 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T07:12:38.101 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T07:12:38.101 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T07:12:38.101 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T07:12:38.101 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T07:12:38.101 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T07:12:38.102 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T07:12:38.103 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T07:12:38.104 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T07:12:38.105 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T07:12:38.106 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T07:12:38.107 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T07:12:38.108 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T07:12:38.109 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T07:12:38.110 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T07:12:38.111 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 3/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 4/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805 5/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 6/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 7/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 8/164 2024-05-09T07:12:40.701 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 9/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 10/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 11/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 12/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 13/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 14/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_ 15/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 16/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 17/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_6 18/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86 19/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 20/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el 21/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9. 22/164 2024-05-09T07:12:40.702 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_6 23/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 24/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 25/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 26/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 27/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 28/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 29/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf. 30/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noa 31/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.n 32/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g968 33/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el 34/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 35/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.e 36/164 2024-05-09T07:12:40.703 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 37/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 38/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 39/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T07:12:40.704 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T07:12:40.705 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T07:12:40.706 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T07:12:40.707 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T07:12:40.708 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T07:12:40.709 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T07:12:40.710 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 149/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 151/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T07:12:40.711 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T07:12:40.712 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout:Upgraded: 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout:Installed: 2024-05-09T07:12:41.000 INFO:teuthology.orchestra.run.smithi188.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.001 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T07:12:41.002 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T07:12:41.003 INFO:teuthology.orchestra.run.smithi188.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T07:12:41.004 INFO:teuthology.orchestra.run.smithi188.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.005 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T07:12:41.006 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T07:12:41.007 INFO:teuthology.orchestra.run.smithi188.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T07:12:41.008 INFO:teuthology.orchestra.run.smithi188.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.009 INFO:teuthology.orchestra.run.smithi188.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T07:12:41.010 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:41.011 INFO:teuthology.orchestra.run.smithi188.stdout:Complete! 2024-05-09T07:12:41.408 DEBUG:teuthology.parallel:result is None 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T07:12:43.168 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.169 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-shell-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.170 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.171 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T07:12:43.172 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T07:12:43.173 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.174 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T07:12:43.175 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T07:12:43.176 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T07:12:43.177 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.178 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T07:12:43.179 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T07:12:43.180 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T07:12:43.181 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T07:12:43.181 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:43.181 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2024-05-09T07:12:43.616 DEBUG:teuthology.parallel:result is None 2024-05-09T07:12:43.617 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:12:43.841 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T07:12:43.863 INFO:teuthology.orchestra.run.smithi160.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T07:12:43.864 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T07:12:43.864 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T07:12:43.866 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-09T07:12:44.092 DEBUG:teuthology.orchestra.run.smithi188:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T07:12:44.118 INFO:teuthology.orchestra.run.smithi188.stdout:19.0.0-2515.g96805dcf.el9 2024-05-09T07:12:44.118 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-09T07:12:44.119 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-09T07:12:44.120 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T07:12:44.121 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:44.121 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T07:12:44.149 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:44.149 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T07:12:44.190 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T07:12:44.190 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:44.190 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T07:12:44.216 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T07:12:44.281 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:44.281 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T07:12:44.309 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T07:12:44.376 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T07:12:44.377 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:44.377 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T07:12:44.402 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T07:12:44.466 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:44.467 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T07:12:44.493 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T07:12:44.558 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T07:12:44.559 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:44.559 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T07:12:44.584 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T07:12:44.649 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:44.649 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T07:12:44.674 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T07:12:44.740 INFO:teuthology.run_tasks:Running task ceph... 2024-05-09T07:12:44.830 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-09T07:12:44.831 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /var/log/ceph 2024-05-09T07:12:44.833 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 777 /var/log/ceph 2024-05-09T07:12:44.859 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-09T07:12:44.860 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T07:12:44.900 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-09T07:12:44.925 INFO:tasks.ceph:Creating extra log directories... 2024-05-09T07:12:44.926 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T07:12:44.965 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-09T07:12:44.997 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-05-09T07:12:44.997 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-05-09T07:12:44.997 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-05-09T07:12:44.997 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:44.997 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T07:12:45.044 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T07:12:45.154 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T07:12:45.178 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:45.179 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-05-09T07:12:45.195 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-05-09T07:12:45.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-05-09T07:12:45.338 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-09T07:12:45.338 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-09T07:12:45.338 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588', 'branch': 'squid', 'description': 'fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7698588', 'kclient-workload': {'sequential': [{'kclient': ['client.1']}, {'workunit': {'clients': {'client.1': ['suites/dbench.sh']}}}]}, 'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'seed': 2334, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '359/512', 'suite': 'fs', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_be85c202969d25af6e130678e7d73a20e9c4ca11/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'be85c202969d25af6e130678e7d73a20e9c4ca11', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi160.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCFRfCdf+YXts8HYJzKiBPmxgnfgXq5jIO0hPvKug4BVVpOaS3i6iuv/eYPmb3Iv3Ks9Z6C6i2vi35dthzAE5sk=', 'smithi188.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHtyB1zHxV3GumUngXv2Sdw2J5t3fzaMi62ldMJeybmnxQ5YqGDg57qhs5T3RKsF613PJ5tgLxbKkA6G0hdQvpY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'parallel': ['user-workload', 'kclient-workload']}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_21:24:04', 'tube': 'smithi', 'user': 'teuthology', 'user-workload': {'sequential': [{'ceph-fuse': ['client.0']}, {'workunit': {'clients': {'client.0': ['suites/iozone.sh']}}}]}, 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-09T07:12:45.339 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T07:12:45.341 DEBUG:teuthology.orchestra.run.smithi188:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-09T07:12:45.388 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T07:12:45.397 DEBUG:teuthology.orchestra.run.smithi188:> sudo install -d -m0777 -- /var/run/ceph 2024-05-09T07:12:45.452 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:45.452 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T07:12:45.476 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T07:12:45.476 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2024-05-09T07:12:45.531 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T07:12:45.531 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:45.531 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 659 Links: 1 2024-05-09T07:12:45.531 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:45.531 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:45.531 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 07:12:35.679387139 +0000 2024-05-09T07:12:45.532 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 07:10:41.184383146 +0000 2024-05-09T07:12:45.532 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 07:10:41.184383146 +0000 2024-05-09T07:12:45.532 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 07:10:41.184383146 +0000 2024-05-09T07:12:45.532 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T07:12:45.596 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T07:12:45.596 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T07:12:45.596 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000336442 s, 1.5 MB/s 2024-05-09T07:12:45.597 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T07:12:45.653 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2024-05-09T07:12:45.710 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T07:12:45.710 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:45.710 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 680 Links: 1 2024-05-09T07:12:45.710 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:45.710 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:45.710 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 07:12:35.679387139 +0000 2024-05-09T07:12:45.711 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 07:10:41.525377800 +0000 2024-05-09T07:12:45.711 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 07:10:41.525377800 +0000 2024-05-09T07:12:45.711 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 07:10:41.525377800 +0000 2024-05-09T07:12:45.711 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T07:12:45.774 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T07:12:45.774 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T07:12:45.774 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000214955 s, 2.4 MB/s 2024-05-09T07:12:45.775 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T07:12:45.831 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 07:12:35.680387124 +0000 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 07:10:41.896371985 +0000 2024-05-09T07:12:45.887 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 07:10:41.896371985 +0000 2024-05-09T07:12:45.888 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 07:10:41.896371985 +0000 2024-05-09T07:12:45.888 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T07:12:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T07:12:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T07:12:45.951 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000184759 s, 2.8 MB/s 2024-05-09T07:12:45.952 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T07:12:46.009 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2024-05-09T07:12:46.065 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T07:12:46.065 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:46.065 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 5h/5d Inode: 718 Links: 1 2024-05-09T07:12:46.065 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:46.065 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:46.066 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2024-05-09 07:12:35.680387124 +0000 2024-05-09T07:12:46.066 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 07:10:42.227366797 +0000 2024-05-09T07:12:46.066 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 07:10:42.227366797 +0000 2024-05-09T07:12:46.066 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: 2024-05-09 07:10:42.227366797 +0000 2024-05-09T07:12:46.066 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T07:12:46.130 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2024-05-09T07:12:46.130 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2024-05-09T07:12:46.130 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000168761 s, 3.0 MB/s 2024-05-09T07:12:46.131 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T07:12:46.187 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T07:12:46.187 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:46.187 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T07:12:46.205 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T07:12:46.205 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_1 2024-05-09T07:12:46.263 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T07:12:46.263 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:46.263 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-09T07:12:46.264 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:46.264 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:46.264 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 07:12:32.998563301 +0000 2024-05-09T07:12:46.264 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 07:10:36.913390267 +0000 2024-05-09T07:12:46.264 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 07:10:36.913390267 +0000 2024-05-09T07:12:46.264 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 07:10:36.913390267 +0000 2024-05-09T07:12:46.264 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T07:12:46.330 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T07:12:46.330 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T07:12:46.330 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.00021856 s, 2.3 MB/s 2024-05-09T07:12:46.331 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T07:12:46.389 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_2 2024-05-09T07:12:46.446 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T07:12:46.446 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:46.446 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-09T07:12:46.446 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:46.446 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:46.447 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 07:12:32.998563301 +0000 2024-05-09T07:12:46.447 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 07:10:37.443380338 +0000 2024-05-09T07:12:46.447 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 07:10:37.443380338 +0000 2024-05-09T07:12:46.447 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 07:10:37.443380338 +0000 2024-05-09T07:12:46.447 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T07:12:46.512 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T07:12:46.512 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T07:12:46.512 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000161359 s, 3.2 MB/s 2024-05-09T07:12:46.513 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T07:12:46.569 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_3 2024-05-09T07:12:46.625 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 07:12:32.999563282 +0000 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 07:10:37.778374063 +0000 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 07:10:37.778374063 +0000 2024-05-09T07:12:46.626 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 07:10:37.778374063 +0000 2024-05-09T07:12:46.626 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T07:12:46.690 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T07:12:46.690 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T07:12:46.690 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000168038 s, 3.0 MB/s 2024-05-09T07:12:46.691 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T07:12:46.748 DEBUG:teuthology.orchestra.run.smithi188:> stat /dev/vg_nvme/lv_4 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 2024-05-09 07:12:32.999563282 +0000 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 07:10:38.127367525 +0000 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 07:10:38.127367525 +0000 2024-05-09T07:12:46.804 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 07:10:38.127367525 +0000 2024-05-09T07:12:46.804 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T07:12:46.867 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records in 2024-05-09T07:12:46.867 INFO:teuthology.orchestra.run.smithi188.stderr:1+0 records out 2024-05-09T07:12:46.867 INFO:teuthology.orchestra.run.smithi188.stderr:512 bytes copied, 0.000196414 s, 2.6 MB/s 2024-05-09T07:12:46.868 DEBUG:teuthology.orchestra.run.smithi188:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T07:12:46.925 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T07:12:46.925 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T07:12:46.925 INFO:tasks.ceph:Generating config... 2024-05-09T07:12:46.926 INFO:tasks.ceph:[client] client mount timeout = 600 2024-05-09T07:12:46.926 INFO:tasks.ceph:[client] debug client = 20 2024-05-09T07:12:46.926 INFO:tasks.ceph:[client] debug ms = 1 2024-05-09T07:12:46.926 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-05-09T07:12:46.926 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-05-09T07:12:46.926 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] debug mds = 20 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] debug ms = 1 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] mds debug frag = True 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mgr] debug client = 20 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-09T07:12:46.927 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-09T07:12:46.928 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-09T07:12:46.928 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-09T07:12:46.928 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-05-09T07:12:46.928 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-09T07:12:46.928 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-09T07:12:46.929 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-09T07:12:46.929 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-09T07:12:46.929 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-05-09T07:12:46.929 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-09T07:12:46.929 INFO:tasks.ceph:Setting up mon.a... 2024-05-09T07:12:46.929 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-09T07:12:46.980 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.keyring 2024-05-09T07:12:46.982 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-09T07:12:47.026 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T07:12:47.051 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.160'), ('mon.b', '172.21.15.188'), ('mon.c', '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]')] 2024-05-09T07:12:47.051 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.160,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-05-09T07:12:47.052 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:47.052 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T07:12:47.107 DEBUG:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.160 --add b 172.21.15.188 --addv c '[v2:172.21.15.188:3301,v1:172.21.15.188:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool: generated fsid 22e9edb5-902d-460d-aabf-1eff035c9624 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:setting min_mon_release = quincy 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:epoch 0 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:fsid 22e9edb5-902d-460d-aabf-1eff035c9624 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:last_changed 2024-05-09T07:12:47.194098+0000 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:created 2024-05-09T07:12:47.194098+0000 2024-05-09T07:12:47.194 INFO:teuthology.orchestra.run.smithi160.stdout:min_mon_release 17 (quincy) 2024-05-09T07:12:47.195 INFO:teuthology.orchestra.run.smithi160.stdout:election_strategy: 1 2024-05-09T07:12:47.195 INFO:teuthology.orchestra.run.smithi160.stdout:0: [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon.a 2024-05-09T07:12:47.195 INFO:teuthology.orchestra.run.smithi160.stdout:1: [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] mon.b 2024-05-09T07:12:47.195 INFO:teuthology.orchestra.run.smithi160.stdout:2: [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] mon.c 2024-05-09T07:12:47.195 INFO:teuthology.orchestra.run.smithi160.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-09T07:12:47.196 DEBUG:teuthology.orchestra.run.smithi160:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-09T07:12:47.211 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 22e9edb5-902d-460d-aabf-1eff035c9624... 2024-05-09T07:12:47.212 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T07:12:47.253 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-09T07:12:47.293 INFO:teuthology.orchestra.run.smithi160.stdout:[global] 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: chdir = "" 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: auth supported = cephx 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: filestore xattr use omap = true 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: mon clock drift allowed = 1.000 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: osd crush chooseleaf type = 0 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: auth debug = true 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on old message = true 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on bug = true 2024-05-09T07:12:47.294 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon pg warn max object skew = 0 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default size = 2 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow primary affinity = true 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd allow pg remap = true 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on legacy crush tunables = false 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on crush straw calc version zero = false 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on no sortbitwise = false 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on osd down out interval zero = false 2024-05-09T07:12:47.295 INFO:teuthology.orchestra.run.smithi160.stdout: mon warn on too few osds = false 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon_allow_pool_size_one = true 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: osd default data pool replay window = 5 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon allow pool delete = true 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon cluster log file level = debug 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: debug asserts on shutdown = true 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon health detail to clog = false 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: mon host = "172.21.15.160,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-05-09T07:12:47.296 INFO:teuthology.orchestra.run.smithi160.stdout: ms die on skipped message = False 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: fsid = 22e9edb5-902d-460d-aabf-1eff035c9624 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout:[osd] 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd journal size = 100 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub load threshold = 5.0 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd scrub max interval = 600 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd mclock profile = high_recovery_ops 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd recover clone overlap = true 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd recovery max chunk = 1048576 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug shutdown = true 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug op order = true 2024-05-09T07:12:47.297 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug verify stray on activate = true 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug trim objects = true 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: osd open classes on start = true 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug pg log writeout = true 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: osd deep scrub update digest min age = 30 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: osd map max advance = 10 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.298 INFO:teuthology.orchestra.run.smithi160.stdout: journal zero on create = true 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: filestore ondisk finisher threads = 3 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: filestore apply finisher threads = 3 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: bdev debug aio = true 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: osd debug misdirected ops = true 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore block size = 96636764160 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: bluestore fsck on mount = True 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluefs = 20 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: debug bluestore = 20 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: debug osd = 20 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: debug rocksdb = 10 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T07:12:47.299 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd full ratio = 0.9 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd nearfull ratio = 0.8 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: osd failsafe full ratio = 0.95 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: osd objectstore = bluestore 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: osd op complaint time = 180 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: osd shutdown pgref assert = True 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr] 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: debug mgr = 20 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2024-05-09T07:12:47.300 INFO:teuthology.orchestra.run.smithi160.stdout: mgr/telemetry/nag = false 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: debug client = 20 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout:[mon] 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: debug mon = 20 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: debug paxos = 20 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: debug auth = 20 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon data avail warn = 5 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon mgr mkfs grace = 240 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min pgs per osd = 4 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd reporter subtree level = osd 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon osd prime pg temp = true 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: mon reweight min bytes per osd = 10 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T07:12:47.301 INFO:teuthology.orchestra.run.smithi160.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: # 1m isn't quite enough 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon_down_mkfs_grace = 2m 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon_warn_on_filestore_osds = false 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon down mkfs grace = 300 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: mon op complaint time = 120 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:12:47.302 INFO:teuthology.orchestra.run.smithi160.stdout:[client] 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: rgw cache enabled = true 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable ops log = true 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: rgw enable usage log = true 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: client mount timeout = 600 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: debug client = 20 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: rados mon op timeout = 900 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout: rados osd op timeout = 900 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.a] 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.b] 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout:[mon.c] 2024-05-09T07:12:47.303 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a] 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c] 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b] 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d] 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout:[mds] 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: debug mds = 20 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: debug mds balancer = 20 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: debug ms = 1 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: mds debug frag = True 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: mds debug scatterstat = True 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: mds op complaint time = 180 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: mds verify scatter = True 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: osd op complaint time = 180 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: rados mon op timeout = 900 2024-05-09T07:12:47.304 INFO:teuthology.orchestra.run.smithi160.stdout: rados osd op timeout = 900 2024-05-09T07:12:47.305 INFO:teuthology.orchestra.run.smithi188.stdout:[global] 2024-05-09T07:12:47.305 INFO:teuthology.orchestra.run.smithi188.stdout: chdir = "" 2024-05-09T07:12:47.305 INFO:teuthology.orchestra.run.smithi188.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-09T07:12:47.305 INFO:teuthology.orchestra.run.smithi188.stdout: auth supported = cephx 2024-05-09T07:12:47.305 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.305 INFO:teuthology.orchestra.run.smithi188.stdout: filestore xattr use omap = true 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: mon clock drift allowed = 1.000 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: osd crush chooseleaf type = 0 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: auth debug = true 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on old message = true 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on bug = true 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: mon pg warn max object skew = 0 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: # disable pg_autoscaler by default for new pools 2024-05-09T07:12:47.306 INFO:teuthology.orchestra.run.smithi188.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default size = 2 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow primary affinity = true 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd allow pg remap = true 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on legacy crush tunables = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on crush straw calc version zero = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on no sortbitwise = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on osd down out interval zero = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon warn on too few osds = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: mon_allow_pool_size_one = true 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.307 INFO:teuthology.orchestra.run.smithi188.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: osd default data pool replay window = 5 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: mon allow pool delete = true 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: mon cluster log file level = debug 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: debug asserts on shutdown = true 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: mon health detail to clog = false 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: mon host = "172.21.15.160,172.21.15.188,[v2:172.21.15.188:3301,v1:172.21.15.188:6790]" 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: ms die on skipped message = False 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: fsid = 22e9edb5-902d-460d-aabf-1eff035c9624 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout:[osd] 2024-05-09T07:12:47.308 INFO:teuthology.orchestra.run.smithi188.stdout: osd journal size = 100 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub load threshold = 5.0 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd scrub max interval = 600 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd mclock profile = high_recovery_ops 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd recover clone overlap = true 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd recovery max chunk = 1048576 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug shutdown = true 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug op order = true 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug verify stray on activate = true 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug trim objects = true 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.309 INFO:teuthology.orchestra.run.smithi188.stdout: osd open classes on start = true 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug pg log writeout = true 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: osd deep scrub update digest min age = 30 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: osd map max advance = 10 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: journal zero on create = true 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: filestore ondisk finisher threads = 3 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: filestore apply finisher threads = 3 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: bdev debug aio = true 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: osd debug misdirected ops = true 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore block size = 96636764160 2024-05-09T07:12:47.310 INFO:teuthology.orchestra.run.smithi188.stdout: bluestore fsck on mount = True 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluefs = 20 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: debug bluestore = 20 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: debug osd = 20 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: debug rocksdb = 10 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd backfillfull_ratio = 0.85 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd full ratio = 0.9 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd nearfull ratio = 0.8 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: osd failsafe full ratio = 0.95 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: osd objectstore = bluestore 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: osd shutdown pgref assert = True 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr] 2024-05-09T07:12:47.311 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug mgr = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: mgr/telemetry/nag = false 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout:[mon] 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug mon = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug paxos = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: debug auth = 20 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: mon data avail warn = 5 2024-05-09T07:12:47.312 INFO:teuthology.orchestra.run.smithi188.stdout: mon mgr mkfs grace = 240 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min pgs per osd = 4 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd reporter subtree level = osd 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon osd prime pg temp = true 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon reweight min bytes per osd = 10 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: auth mon ticket ttl = 660 # 11m 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: auth service ticket ttl = 240 # 4m 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: # don't complain about insecure global_id in the test suite 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: # 1m isn't quite enough 2024-05-09T07:12:47.313 INFO:teuthology.orchestra.run.smithi188.stdout: mon_down_mkfs_grace = 2m 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: mon_warn_on_filestore_osds = false 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: mon down mkfs grace = 300 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: mon op complaint time = 120 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout:[client] 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: rgw cache enabled = true 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable ops log = true 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: rgw enable usage log = true 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: client mount timeout = 600 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: debug client = 20 2024-05-09T07:12:47.314 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.a] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.b] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mon.c] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout:[mds] 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds = 20 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: debug mds balancer = 20 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: debug ms = 1 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug frag = True 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: mds debug scatterstat = True 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: mds op complaint time = 180 2024-05-09T07:12:47.315 INFO:teuthology.orchestra.run.smithi188.stdout: mds verify scatter = True 2024-05-09T07:12:47.316 INFO:teuthology.orchestra.run.smithi188.stdout: osd op complaint time = 180 2024-05-09T07:12:47.316 INFO:teuthology.orchestra.run.smithi188.stdout: rados mon op timeout = 900 2024-05-09T07:12:47.316 INFO:teuthology.orchestra.run.smithi188.stdout: rados osd op timeout = 900 2024-05-09T07:12:47.317 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-09T07:12:47.317 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-09T07:12:47.411 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-09T07:12:47.412 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:47.412 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-09T07:12:47.427 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:47.427 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-09T07:12:47.483 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:12:47.484 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:47.484 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T07:12:47.484 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T07:12:47.558 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:12:47.558 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:47.613 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:12:47.613 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:47.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-09T07:12:47.613 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-09T07:12:47.648 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:12:47.648 DEBUG:teuthology.orchestra.run.smithi188:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:47.703 INFO:tasks.ceph:Setting up mon nodes... 2024-05-09T07:12:47.703 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-09T07:12:47.703 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T07:12:47.754 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-09T07:12:47.757 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T07:12:47.830 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-09T07:12:47.833 INFO:tasks.ceph:Setting up mds nodes... 2024-05-09T07:12:47.833 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T07:12:47.884 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-05-09T07:12:47.886 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-05-09T07:12:47.911 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T07:12:48.003 INFO:teuthology.orchestra.run.smithi160.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-05-09T07:12:48.006 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-05-09T07:12:48.031 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T07:12:48.083 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-05-09T07:12:48.085 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-05-09T07:12:48.110 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T07:12:48.200 INFO:teuthology.orchestra.run.smithi188.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-05-09T07:12:48.203 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-05-09T07:12:48.227 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-09T07:12:48.227 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T07:12:48.267 INFO:teuthology.orchestra.run.smithi160.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-09T07:12:48.281 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-09T07:12:48.322 INFO:teuthology.orchestra.run.smithi188.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-09T07:12:48.335 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-09T07:12:48.335 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi160.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi188.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-09T07:12:48.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-09T07:12:48.359 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T07:12:48.359 INFO:tasks.ceph:role: osd.0 2024-05-09T07:12:48.359 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:12:48.359 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T07:12:48.431 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:12:48.432 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:12:48.931 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-05-09T07:12:48.933 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-05-09T07:12:48.933 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-09T07:12:49.066 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-09T07:12:49.101 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-09T07:12:49.167 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T07:12:49.168 INFO:tasks.ceph:role: osd.1 2024-05-09T07:12:49.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:12:49.168 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T07:12:49.240 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:12:49.240 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:12:49.240 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:12:49.241 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:12:49.710 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-05-09T07:12:49.712 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-05-09T07:12:49.712 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-09T07:12:49.798 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-09T07:12:49.827 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-09T07:12:49.893 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T07:12:49.893 INFO:tasks.ceph:role: osd.2 2024-05-09T07:12:49.893 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:12:49.893 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:12:49.966 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:12:50.450 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-05-09T07:12:50.453 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-05-09T07:12:50.453 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-09T07:12:50.585 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-09T07:12:50.614 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-09T07:12:50.679 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-09T07:12:50.680 INFO:tasks.ceph:role: osd.3 2024-05-09T07:12:50.680 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:12:50.680 DEBUG:teuthology.orchestra.run.smithi160:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T07:12:50.752 INFO:teuthology.orchestra.run.smithi160.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:12:50.752 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:12:50.752 INFO:teuthology.orchestra.run.smithi160.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:12:50.752 INFO:teuthology.orchestra.run.smithi160.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:12:50.752 INFO:teuthology.orchestra.run.smithi160.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:12:50.753 INFO:teuthology.orchestra.run.smithi160.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:12:50.753 INFO:teuthology.orchestra.run.smithi160.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:12:50.753 INFO:teuthology.orchestra.run.smithi160.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:12:50.753 INFO:teuthology.orchestra.run.smithi160.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:12:50.753 INFO:teuthology.orchestra.run.smithi160.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:12:51.285 INFO:teuthology.orchestra.run.smithi160.stdout:Discarding blocks...Done. 2024-05-09T07:12:51.287 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi160.front.sepia.ceph.com -o noatime 2024-05-09T07:12:51.287 DEBUG:teuthology.orchestra.run.smithi160:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-09T07:12:51.432 DEBUG:teuthology.orchestra.run.smithi160:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-09T07:12:51.465 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:51.581+0000 7feb3880d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-09T07:12:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:51.581+0000 7feb3880d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-09T07:12:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:51.581+0000 7feb3880d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T07:12:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:51.581+0000 7feb3880d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T07:12:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:51.581+0000 7feb3880d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-09T07:12:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:51.582+0000 7feb3880d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-09T07:12:54.437 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-09T07:12:54.466 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:54.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:54.556+0000 7efcada0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-09T07:12:54.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:54.556+0000 7efcada0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-09T07:12:54.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:54.556+0000 7efcada0f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T07:12:54.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:54.556+0000 7efcada0f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T07:12:54.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:54.556+0000 7efcada0f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-09T07:12:54.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:54.556+0000 7efcada0f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-09T07:12:57.406 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-09T07:12:57.437 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:12:57.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:57.527+0000 7f2589e0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-09T07:12:57.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:57.527+0000 7f2589e0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-09T07:12:57.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:57.528+0000 7f2589e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T07:12:57.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:57.528+0000 7f2589e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T07:12:57.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:57.528+0000 7f2589e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-09T07:12:57.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:12:57.528+0000 7f2589e0f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-09T07:13:00.388 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-09T07:13:00.418 DEBUG:teuthology.orchestra.run.smithi160:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:00.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:00.508+0000 7fd40660d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-09T07:13:00.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:00.508+0000 7fd40660d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-09T07:13:00.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:00.508+0000 7fd40660d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T07:13:00.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:00.508+0000 7fd40660d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T07:13:00.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:00.508+0000 7fd40660d640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-09T07:13:00.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:00.509+0000 7fd40660d640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-09T07:13:03.368 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-09T07:13:03.398 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-09T07:13:03.428 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T07:13:03.428 INFO:tasks.ceph:role: osd.4 2024-05-09T07:13:03.428 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:13:03.428 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:13:03.505 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:13:03.987 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-09T07:13:03.989 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-09T07:13:03.989 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-09T07:13:04.086 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-09T07:13:04.121 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-09T07:13:04.187 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T07:13:04.187 INFO:tasks.ceph:role: osd.5 2024-05-09T07:13:04.187 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:13:04.187 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-09T07:13:04.260 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:13:04.260 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:13:04.260 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:13:04.261 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:13:04.706 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-09T07:13:04.709 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-09T07:13:04.709 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-09T07:13:04.782 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-09T07:13:04.851 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-09T07:13:04.915 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T07:13:04.916 INFO:tasks.ceph:role: osd.6 2024-05-09T07:13:04.916 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:13:04.916 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:13:04.988 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:13:04.989 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:13:05.467 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-09T07:13:05.469 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-09T07:13:05.470 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-09T07:13:05.577 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-09T07:13:05.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-09T07:13:05.670 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-09T07:13:05.671 INFO:tasks.ceph:role: osd.7 2024-05-09T07:13:05.671 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:13:05.671 DEBUG:teuthology.orchestra.run.smithi188:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout: = sunit=0 swidth=0 blks 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-09T07:13:05.744 INFO:teuthology.orchestra.run.smithi188.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-09T07:13:06.255 INFO:teuthology.orchestra.run.smithi188.stdout:Discarding blocks...Done. 2024-05-09T07:13:06.257 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi188.front.sepia.ceph.com -o noatime 2024-05-09T07:13:06.257 DEBUG:teuthology.orchestra.run.smithi188:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-09T07:13:06.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-09T07:13:06.411 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:06.533 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:06.530+0000 7f4c3cc43640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-09T07:13:06.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:06.535+0000 7f4c3cc43640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-09T07:13:06.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:06.535+0000 7f4c3cc43640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T07:13:06.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:06.535+0000 7f4c3cc43640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T07:13:06.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:06.535+0000 7f4c3cc43640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-09T07:13:06.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:06.542+0000 7f4c3cc43640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-09T07:13:09.387 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-09T07:13:09.414 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:09.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:09.498+0000 7fbad6d33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-09T07:13:09.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:09.499+0000 7fbad6d33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-09T07:13:09.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:09.499+0000 7fbad6d33640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T07:13:09.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:09.499+0000 7fbad6d33640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T07:13:09.501 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:09.499+0000 7fbad6d33640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-09T07:13:09.502 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:09.499+0000 7fbad6d33640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-09T07:13:12.341 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-09T07:13:12.371 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:12.461 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:12.460+0000 7f0c0a663640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-09T07:13:12.462 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:12.460+0000 7f0c0a663640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-09T07:13:12.462 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:12.460+0000 7f0c0a663640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T07:13:12.462 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:12.460+0000 7f0c0a663640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T07:13:12.462 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:12.460+0000 7f0c0a663640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-09T07:13:12.462 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:12.461+0000 7f0c0a663640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-09T07:13:15.307 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-09T07:13:15.332 DEBUG:teuthology.orchestra.run.smithi188:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:15.340 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:13:15.343 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:13:15.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:15.417+0000 7f0086014640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-09T07:13:15.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:15.417+0000 7f0086014640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-09T07:13:15.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:15.417+0000 7f0086014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T07:13:15.419 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:15.417+0000 7f0086014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T07:13:15.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:15.417+0000 7f0086014640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-09T07:13:15.420 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:15.418+0000 7f0086014640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-09T07:13:18.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-09T07:13:18.296 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-09T07:13:18.297 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.297 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-09T07:13:18.327 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.327 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-05-09T07:13:18.395 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.395 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-05-09T07:13:18.464 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.464 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-09T07:13:18.531 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.531 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-09T07:13:18.597 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.597 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-09T07:13:18.664 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:18.664 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-09T07:13:18.730 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:18.731 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-09T07:13:18.760 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:18.760 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-05-09T07:13:18.829 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:18.829 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-05-09T07:13:18.897 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:18.897 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-09T07:13:18.964 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:18.964 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-09T07:13:19.030 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:19.030 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-09T07:13:19.096 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:19.096 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-09T07:13:19.162 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:19.162 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-09T07:13:19.179 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:19.179 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-09T07:13:19.218 INFO:tasks.ceph:Adding keys to all mons... 2024-05-09T07:13:19.218 DEBUG:teuthology.orchestra.run.smithi160:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T07:13:19.222 DEBUG:teuthology.orchestra.run.smithi188:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-09T07:13:19.262 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.y] 2024-05-09T07:13:19.262 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBvdzxmklr7LBAAylmcVEiCNZKMXeJGeokVbw== 2024-05-09T07:13:19.262 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.a] 2024-05-09T07:13:19.262 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBvdzxmVBG3NBAAA+AYcuhIggHyRJojffX1Jw== 2024-05-09T07:13:19.262 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.c] 2024-05-09T07:13:19.263 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBwdzxmKrI8ABAAGjIOfg2kaWStTRPUv8Q3pg== 2024-05-09T07:13:19.263 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.0] 2024-05-09T07:13:19.263 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBzdzxmici4IhAAkqI5vfVzG/f7tWS/er/lnA== 2024-05-09T07:13:19.263 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.1] 2024-05-09T07:13:19.263 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB2dzxmhPcyIRAAjXNgEa1xZ1GvyendlpyB1Q== 2024-05-09T07:13:19.264 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.2] 2024-05-09T07:13:19.264 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB5dzxmnV6AHxAAuP3WG9oxBy7DydIXF6dgjA== 2024-05-09T07:13:19.264 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.3] 2024-05-09T07:13:19.264 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQB8dzxmsdVbHhAAV/OmCgFUGgRJFZe/kT/GVg== 2024-05-09T07:13:19.264 INFO:teuthology.orchestra.run.smithi160.stdout:[mgr.x] 2024-05-09T07:13:19.264 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBvdzxmTKSAMRAACNlHYkHUCOwPqTL7qkyp4Q== 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.b] 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBwdzxmbFf1BBAAA5OvyG7S7Kd32VISEpU3qA== 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout:[mds.d] 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBwdzxmGbn2CxAAVRMER7IznqltK+yW6Pzr7Q== 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.4] 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCCdzxmZlLCHxAAqBCB2dm5zcYZq7KT4wvmAw== 2024-05-09T07:13:19.265 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.5] 2024-05-09T07:13:19.266 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCFdzxmzgTZHRAAtKufiPDJVsatAqktycdDyw== 2024-05-09T07:13:19.266 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.6] 2024-05-09T07:13:19.266 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCIdzxmmC9/GxAA4lxVV1vOd7YkspOkM8XkBQ== 2024-05-09T07:13:19.266 INFO:teuthology.orchestra.run.smithi160.stdout:[osd.7] 2024-05-09T07:13:19.266 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQCLdzxmhVPvGBAAN48fGLfy7S56DlOg7BDHaA== 2024-05-09T07:13:19.267 INFO:teuthology.orchestra.run.smithi160.stdout:[client.0] 2024-05-09T07:13:19.267 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBwdzxmPtr3DxAAxxe1sifahwLbk01NTpgoLA== 2024-05-09T07:13:19.267 INFO:teuthology.orchestra.run.smithi160.stdout:[client.1] 2024-05-09T07:13:19.267 INFO:teuthology.orchestra.run.smithi160.stdout: key = AQBwdzxmr286ExAAaI4XSDv9fMm7GKxUIf8iMw== 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.y] 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBvdzxmklr7LBAAylmcVEiCNZKMXeJGeokVbw== 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.a] 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBvdzxmVBG3NBAAA+AYcuhIggHyRJojffX1Jw== 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.c] 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwdzxmKrI8ABAAGjIOfg2kaWStTRPUv8Q3pg== 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.0] 2024-05-09T07:13:19.281 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBzdzxmici4IhAAkqI5vfVzG/f7tWS/er/lnA== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.1] 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQB2dzxmhPcyIRAAjXNgEa1xZ1GvyendlpyB1Q== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.2] 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQB5dzxmnV6AHxAAuP3WG9oxBy7DydIXF6dgjA== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.3] 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQB8dzxmsdVbHhAAV/OmCgFUGgRJFZe/kT/GVg== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[mgr.x] 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBvdzxmTKSAMRAACNlHYkHUCOwPqTL7qkyp4Q== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.b] 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwdzxmbFf1BBAAA5OvyG7S7Kd32VISEpU3qA== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[mds.d] 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwdzxmGbn2CxAAVRMER7IznqltK+yW6Pzr7Q== 2024-05-09T07:13:19.282 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.4] 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCCdzxmZlLCHxAAqBCB2dm5zcYZq7KT4wvmAw== 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.5] 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCFdzxmzgTZHRAAtKufiPDJVsatAqktycdDyw== 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.6] 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCIdzxmmC9/GxAA4lxVV1vOd7YkspOkM8XkBQ== 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout:[osd.7] 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQCLdzxmhVPvGBAAN48fGLfy7S56DlOg7BDHaA== 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout:[client.0] 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwdzxmPtr3DxAAxxe1sifahwLbk01NTpgoLA== 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout:[client.1] 2024-05-09T07:13:19.283 INFO:teuthology.orchestra.run.smithi188.stdout: key = AQBwdzxmr286ExAAaI4XSDv9fMm7GKxUIf8iMw== 2024-05-09T07:13:19.284 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T07:13:19.313 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T07:13:19.376 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.378 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.432 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.488 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.491 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.541 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.544 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.594 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.596 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.646 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.649 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.700 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T07:13:19.702 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-09T07:13:19.752 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.754 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.808 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.810 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-05-09T07:13:19.863 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.866 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.916 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.918 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.969 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:19.971 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:20.022 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:20.024 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-09T07:13:20.075 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T07:13:20.077 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T07:13:20.131 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T07:13:20.133 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-09T07:13:20.186 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-09T07:13:20.186 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-09T07:13:20.211 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T07:13:20.307 DEBUG:teuthology.orchestra.run.smithi160:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-09T07:13:20.331 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-09T07:13:20.356 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T07:13:20.451 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-09T07:13:20.476 DEBUG:teuthology.orchestra.run.smithi188:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-09T07:13:20.540 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-09T07:13:20.638 DEBUG:teuthology.orchestra.run.smithi188:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-09T07:13:20.662 DEBUG:teuthology.orchestra.run.smithi160:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:20.664 DEBUG:teuthology.orchestra.run.smithi188:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-09T07:13:20.717 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-09T07:13:20.718 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-09T07:13:20.718 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-09T07:13:20.721 INFO:tasks.ceph.mon.a:Started 2024-05-09T07:13:20.722 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-09T07:13:20.722 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-09T07:13:20.760 INFO:tasks.ceph.mon.b:Started 2024-05-09T07:13:20.760 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-09T07:13:20.761 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-09T07:13:20.762 INFO:tasks.ceph.mon.c:Started 2024-05-09T07:13:20.763 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-09T07:13:20.763 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-09T07:13:20.763 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-09T07:13:20.765 INFO:tasks.ceph.mgr.y:Started 2024-05-09T07:13:20.765 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-09T07:13:20.765 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-09T07:13:20.767 INFO:tasks.ceph.mgr.x:Started 2024-05-09T07:13:20.767 INFO:tasks.ceph:Setting crush tunables to default 2024-05-09T07:13:20.767 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph osd crush tunables default 2024-05-09T07:13:20.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.866+0000 7fe69ac2a640 1 Processor -- start 2024-05-09T07:13:20.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe69ac2a640 1 -- start start 2024-05-09T07:13:20.869 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe69ac2a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 0x7fe694152110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe69ac2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe69415c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe69ac2a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe69406f930 con 0x7fe6940735f0 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe69ac2a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe6940c4210 con 0x7fe6940c7d00 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe69ac2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe6940c8530 con 0x7fe6940c80e0 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe693fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe6940735f0 0x7fe6940c3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:49562/0 (socket says 172.21.15.160:49562) 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe693fff640 1 -- 172.21.15.160:0/1898542616 learned_addr learned my addr 172.21.15.160:0/1898542616 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.867+0000 7fe6991c7640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe69415c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 0x7fe694152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6991c7640 1 -- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 msgr2=0x7fe69415c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6991c7640 1 -- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 msgr2=0x7fe69415c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6991c7640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe69415c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6991c7640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe69415c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T07:13:20.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6989c6640 1 -- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 msgr2=0x7fe694152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-05-09T07:13:20.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6989c6640 1 -- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 msgr2=0x7fe694152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-05-09T07:13:20.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 0x7fe694152110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T07:13:20.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:20.868+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 0x7fe694152110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 47185735 0 0) 0x7fe69406f930 con 0x7fe6940735f0 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe684005730 con 0x7fe6940735f0 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2910999555 0 0) 0x7fe684005730 con 0x7fe6940735f0 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 msgr2=0x7fe69415c9d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe69415c9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 msgr2=0x7fe694152110 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 --2- 172.21.15.160:0/1898542616 >> v2:172.21.15.188:3300/0 conn(0x7fe6940c7d00 0x7fe694152110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe694156230 con 0x7fe6940735f0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7fe680001670 con 0x7fe6940735f0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 >> v1:172.21.15.160:6789/0 conn(0x7fe6940735f0 legacy=0x7fe6940c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe684005bc0 0x7fe684006000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe684006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe684006f50 0x7fe6840073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940c4210 con 0x7fe684005bc0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940c8530 con 0x7fe684006f50 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe68400dde0 con 0x7fe6940c80e0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe684005bc0 0x7fe684006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe6991c7640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe684006f50 0x7fe6840073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.025+0000 7fe693fff640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe684006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe6989c6640 1 -- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 msgr2=0x7fe684006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:21.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe684006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe6989c6640 1 -- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe684006f50 msgr2=0x7fe6840073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe684006f50 0x7fe6840073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe6989c6640 1 -- 172.21.15.160:0/1898542616 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe684005730 con 0x7fe684005bc0 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe693fff640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe6940c80e0 0x7fe684006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.026+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe684005bc0 0x7fe684006000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b9e0 tx=0x7fe684007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf2f04d4f34e417 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.027+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688030070 con 0x7fe684005bc0 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.027+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe68802adb0 con 0x7fe684005bc0 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.027+0000 7fe6937fe640 1 -- 172.21.15.160:0/1898542616 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68800b5e0 con 0x7fe684005bc0 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.027+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1898542616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe684005bc0 msgr2=0x7fe684006000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.027+0000 7fe69ac2a640 1 --2- 172.21.15.160:0/1898542616 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe684005bc0 0x7fe684006000 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b9e0 tx=0x7fe684007a40 comp rx=0 tx=0).stop 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.027+0000 7fe6991c7640 1 -- 172.21.15.160:0/1898542616 reap_dead start 2024-05-09T07:13:21.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.028+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1898542616 shutdown_connections 2024-05-09T07:13:21.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.028+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1898542616 >> 172.21.15.160:0/1898542616 conn(0x7fe69406c450 msgr2=0x7fe69406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:21.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.028+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1898542616 shutdown_connections 2024-05-09T07:13:21.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.028+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1898542616 wait complete. 2024-05-09T07:13:21.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.029+0000 7fe69ac2a640 1 Processor -- start 2024-05-09T07:13:21.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.029+0000 7fe69ac2a640 1 -- start start 2024-05-09T07:13:21.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe69ac2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe684005bc0 0x7fe6941e6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe69ac2a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 0x7fe6941dddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe69ac2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6941e8060 0x7fe69414ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe69ac2a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe6940730a0 con 0x7fe6941dcb50 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe69ac2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe694156510 con 0x7fe6941e8060 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe69ac2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe694073220 con 0x7fe684005bc0 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe693fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 0x7fe6941dddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe693fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 0x7fe6941dddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:49306/0 (socket says 172.21.15.160:49306) 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe693fff640 1 -- 172.21.15.160:0/1138272233 learned_addr learned my addr 172.21.15.160:0/1138272233 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe6991c7640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6941e8060 0x7fe69414ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.030+0000 7fe6989c6640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe684005bc0 0x7fe6941e6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe693fff640 1 -- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe684005bc0 msgr2=0x7fe6941e6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe693fff640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe684005bc0 0x7fe6941e6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe693fff640 1 -- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6941e8060 msgr2=0x7fe69414ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe693fff640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6941e8060 0x7fe69414ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe693fff640 1 -- 172.21.15.160:0/1138272233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6940acb00 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe6991c7640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6941e8060 0x7fe69414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe693fff640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 0x7fe6941dddf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe68000e820 tx=0x7fe68000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68000c870 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe680018070 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6941dbab0 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68000cc50 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941dc070 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe68001d460 con 0x7fe6941dcb50 2024-05-09T07:13:21.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.031+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe680013650 con 0x7fe6941dcb50 2024-05-09T07:13:21.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.032+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe694075650 con 0x7fe6941dcb50 2024-05-09T07:13:21.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.034+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe68000cdf0 con 0x7fe6941dcb50 2024-05-09T07:13:21.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.120+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe69406f670 con 0x7fe6941dcb50 2024-05-09T07:13:21.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.122+0000 7fe6917fa640 1 -- 172.21.15.160:0/1138272233 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fe68001d8e0 con 0x7fe6941dcb50 2024-05-09T07:13:21.123 INFO:teuthology.orchestra.run.smithi160.stderr:adjusted tunables profile to default 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.122+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 msgr2=0x7fe6941dddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.122+0000 7fe69ac2a640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 0x7fe6941dddf0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe68000e820 tx=0x7fe68000ece0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 shutdown_connections 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe6941e8060 0x7fe69414ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe6941dcb50 0x7fe6941dddf0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 --2- 172.21.15.160:0/1138272233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe684005bc0 0x7fe6941e6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 >> 172.21.15.160:0/1138272233 conn(0x7fe69406c450 msgr2=0x7fe694071920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:21.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 shutdown_connections 2024-05-09T07:13:21.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:21.123+0000 7fe69ac2a640 1 -- 172.21.15.160:0/1138272233 wait complete. 2024-05-09T07:13:21.135 INFO:tasks.ceph:check_enable_crimson: False 2024-05-09T07:13:21.135 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-09T07:13:21.135 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:21.135 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-09T07:13:21.144 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:21.143+0000 7f0b0ec12040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T07:13:21.153 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:21.152+0000 7f5cb480d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T07:13:21.160 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:21.161 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-09T07:13:21.228 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:21.228 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-09T07:13:21.293 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:21.293 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-09T07:13:21.307 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:21.305+0000 7f0b0ec12040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T07:13:21.320 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:21.318+0000 7f5cb480d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T07:13:21.344 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:21.344 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-09T07:13:21.360 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:21.359+0000 7f0b0ec12040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T07:13:21.369 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:21.369 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-09T07:13:21.375 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:21.374+0000 7f5cb480d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T07:13:21.434 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:21.434 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-09T07:13:21.499 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:13:21.499 DEBUG:teuthology.orchestra.run.smithi188:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-09T07:13:21.564 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 64c086fb-92b0-426f-88dc-821957a5704b 0 2024-05-09T07:13:21.608 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:21.607+0000 7f0b0ec12040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T07:13:21.622 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:21.621+0000 7f5cb480d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T07:13:21.656 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:21.655+0000 7f0b0ec12040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T07:13:21.667 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:21.665+0000 7f5cb480d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T07:13:21.704 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.703+0000 7fb22b9a9640 1 Processor -- start 2024-05-09T07:13:21.705 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.703+0000 7fb22b9a9640 1 -- start start 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb22b9a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb224153d10 0x7fb2240ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb22b9a9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb2240c84e0 0x7fb2240a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb22b9a9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb2240c6b00 con 0x7fb2240c4810 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb22b9a9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb224154210 con 0x7fb2240c84e0 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb22b9a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2240c6de0 con 0x7fb224153d10 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb228f44640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb224153d10 0x7fb2240ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.706 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb229745640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fb2240c4810 0x7fb2240acb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:51162/0 (socket says 172.21.15.188:51162) 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb229745640 1 -- 172.21.15.188:0/3268261696 learned_addr learned my addr 172.21.15.188:0/3268261696 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb229f46640 1 --2- 172.21.15.188:0/3268261696 >> v2:172.21.15.160:3300/0 conn(0x7fb2240c84e0 0x7fb2240a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.704+0000 7fb217fff640 1 -- 172.21.15.188:0/3268261696 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2255870419 0 0) 0x7fb2240c6b00 con 0x7fb2240c4810 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.705+0000 7fb228f44640 1 -- 172.21.15.188:0/3268261696 >> v2:172.21.15.160:3300/0 conn(0x7fb2240c84e0 msgr2=0x7fb2240a7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.705+0000 7fb228f44640 1 --2- 172.21.15.188:0/3268261696 >> v2:172.21.15.160:3300/0 conn(0x7fb2240c84e0 0x7fb2240a7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.705+0000 7fb228f44640 1 -- 172.21.15.188:0/3268261696 >> v1:172.21.15.188:6789/0 conn(0x7fb2240c4810 legacy=0x7fb2240acb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.707 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.705+0000 7fb228f44640 1 -- 172.21.15.188:0/3268261696 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2240c70c0 con 0x7fb224153d10 2024-05-09T07:13:21.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb228f44640 1 --2- 172.21.15.188:0/3268261696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb224153d10 0x7fb2240ad220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2180097a0 tx=0x7fb21802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab3b0890b4875d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:21.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb217fff640 1 -- 172.21.15.188:0/3268261696 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218037070 con 0x7fb224153d10 2024-05-09T07:13:21.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb217fff640 1 -- 172.21.15.188:0/3268261696 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb21802ae50 con 0x7fb224153d10 2024-05-09T07:13:21.708 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb217fff640 1 -- 172.21.15.188:0/3268261696 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb218032660 con 0x7fb224153d10 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3268261696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb224153d10 msgr2=0x7fb2240ad220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3268261696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb224153d10 0x7fb2240ad220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb2180097a0 tx=0x7fb21802a100 comp rx=0 tx=0).stop 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3268261696 shutdown_connections 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3268261696 >> v2:172.21.15.160:3300/0 conn(0x7fb2240c84e0 0x7fb2240a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3268261696 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb224153d10 0x7fb2240ad220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.706+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3268261696 >> 172.21.15.188:0/3268261696 conn(0x7fb2240c8ad0 msgr2=0x7fb2240c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.707+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3268261696 shutdown_connections 2024-05-09T07:13:21.709 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.707+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3268261696 wait complete. 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 Processor -- start 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 -- start start 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 0x7fb2240bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2240c84e0 0x7fb2241e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb224153d10 0x7fb2241e62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb2240c4bf0 con 0x7fb2240c84e0 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb2240c78c0 con 0x7fb224153d10 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb22b9a9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb2240c4d70 con 0x7fb2240c4810 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb229745640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 0x7fb2240bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb229745640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 0x7fb2240bdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52278/0 (socket says 172.21.15.188:52278) 2024-05-09T07:13:21.712 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb229f46640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb224153d10 0x7fb2241e62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.708+0000 7fb229745640 1 -- 172.21.15.188:0/3199076132 learned_addr learned my addr 172.21.15.188:0/3199076132 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb228f44640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2240c84e0 0x7fb2241e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb229745640 1 -- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb224153d10 msgr2=0x7fb2241e62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb229745640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb224153d10 0x7fb2241e62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb229745640 1 -- 172.21.15.188:0/3199076132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2240c84e0 msgr2=0x7fb2241e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb229745640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2240c84e0 0x7fb2241e5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb229745640 1 -- 172.21.15.188:0/3199076132 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2240c2440 con 0x7fb2240c4810 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb229745640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 0x7fb2240bdef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb22000c460 tx=0x7fb22000c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220018070 con 0x7fb2240c4810 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb220007500 con 0x7fb2240c4810 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2240c2720 con 0x7fb2240c4810 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb220004a20 con 0x7fb2240c4810 2024-05-09T07:13:21.713 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb22001b070 con 0x7fb2240c4810 2024-05-09T07:13:21.714 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.709+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2240c2ce0 con 0x7fb2240c4810 2024-05-09T07:13:21.714 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.710+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fb22001ccd0 con 0x7fb2240c4810 2024-05-09T07:13:21.714 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.710+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb224075650 con 0x7fb2240c4810 2024-05-09T07:13:21.714 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.712+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb220022020 con 0x7fb2240c4810 2024-05-09T07:13:21.787 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.786+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "64c086fb-92b0-426f-88dc-821957a5704b", "id": 0} v 0) -- 0x7fb2240c63d0 con 0x7fb2240c4810 2024-05-09T07:13:21.792 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.791+0000 7fb215ffb640 1 -- 172.21.15.188:0/3199076132 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "64c086fb-92b0-426f-88dc-821957a5704b", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fb22001c3b0 con 0x7fb2240c4810 2024-05-09T07:13:21.793 INFO:teuthology.orchestra.run.smithi188.stdout:0 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 msgr2=0x7fb2240bdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 0x7fb2240bdef0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb22000c460 tx=0x7fb22000c920 comp rx=0 tx=0).stop 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 shutdown_connections 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb224153d10 0x7fb2241e62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2240c84e0 0x7fb2241e5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 --2- 172.21.15.188:0/3199076132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb2240c4810 0x7fb2240bdef0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 >> 172.21.15.188:0/3199076132 conn(0x7fb2240c8ad0 msgr2=0x7fb22414e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:21.794 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 shutdown_connections 2024-05-09T07:13:21.795 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.792+0000 7fb22b9a9640 1 -- 172.21.15.188:0/3199076132 wait complete. 2024-05-09T07:13:21.805 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 348cc74d-e143-48f9-a79f-225bd020ac74 1 2024-05-09T07:13:21.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 Processor -- start 2024-05-09T07:13:21.945 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 -- start start 2024-05-09T07:13:21.947 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed40c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.947 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 0x7f1ed415dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.947 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1ed40c7620 con 0x7f1ed406fac0 2024-05-09T07:13:21.947 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1ed40714e0 con 0x7f1ed40c4690 2024-05-09T07:13:21.947 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.944+0000 7f1eda6ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed40c78c0 con 0x7f1ed41573a0 2024-05-09T07:13:21.947 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed40c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed8c57640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 0x7f1ed415dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed40c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52280/0 (socket says 172.21.15.188:52280) 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed8c57640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 0x7f1ed415dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53492/0 (socket says 172.21.15.188:53492) 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 -- 172.21.15.188:0/1412950638 learned_addr learned my addr 172.21.15.188:0/1412950638 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 -- 172.21.15.188:0/1412950638 >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 msgr2=0x7f1ed415dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 --2- 172.21.15.188:0/1412950638 >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 0x7f1ed415dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 -- 172.21.15.188:0/1412950638 >> v1:172.21.15.160:6789/0 conn(0x7f1ed406fac0 legacy=0x7f1ed40c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed3fff640 1 -- 172.21.15.188:0/1412950638 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed4071200 con 0x7f1ed41573a0 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.945+0000 7f1ed8c57640 1 --2- 172.21.15.188:0/1412950638 >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 0x7f1ed415dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1ed3fff640 1 --2- 172.21.15.188:0/1412950638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed40c8ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009870 tx=0x7f1ec402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8643ffdb9ba88b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1ed2ffd640 1 -- 172.21.15.188:0/1412950638 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4037070 con 0x7f1ed41573a0 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1ed2ffd640 1 -- 172.21.15.188:0/1412950638 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ec402ae50 con 0x7f1ed41573a0 2024-05-09T07:13:21.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1ed2ffd640 1 -- 172.21.15.188:0/1412950638 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4032860 con 0x7f1ed41573a0 2024-05-09T07:13:21.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 -- 172.21.15.188:0/1412950638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 msgr2=0x7f1ed40c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/1412950638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed40c8ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009870 tx=0x7f1ec402a300 comp rx=0 tx=0).stop 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 -- 172.21.15.188:0/1412950638 shutdown_connections 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/1412950638 >> v2:172.21.15.188:3300/0 conn(0x7f1ed40c4690 0x7f1ed415dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/1412950638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed40c8ae0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 -- 172.21.15.188:0/1412950638 >> 172.21.15.188:0/1412950638 conn(0x7f1ed4072a80 msgr2=0x7f1ed4072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 -- 172.21.15.188:0/1412950638 shutdown_connections 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.946+0000 7f1eda6ba640 1 -- 172.21.15.188:0/1412950638 wait complete. 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.947+0000 7f1eda6ba640 1 Processor -- start 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 -- start start 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 0x7f1ed4155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed40c4690 0x7f1ed414ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed414b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed40abc20 con 0x7f1ed40c4690 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed40c3af0 con 0x7f1ed406fac0 2024-05-09T07:13:21.952 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1eda6ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed4070520 con 0x7f1ed41573a0 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 0x7f1ed4155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.948+0000 7f1ed8c57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed414b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed3fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 0x7f1ed4155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53504/0 (socket says 172.21.15.188:53504) 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed414b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52290/0 (socket says 172.21.15.188:52290) 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed3fff640 1 -- 172.21.15.188:0/2462266866 learned_addr learned my addr 172.21.15.188:0/2462266866 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed37fe640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed40c4690 0x7f1ed414ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 -- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 msgr2=0x7f1ed4155df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 0x7f1ed4155df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 -- 172.21.15.188:0/2462266866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed40c4690 msgr2=0x7f1ed414ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed40c4690 0x7f1ed414ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 -- 172.21.15.188:0/2462266866 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41e8ea0 con 0x7f1ed41573a0 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed3fff640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 0x7f1ed4155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed8c57640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed414b3d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00efc0 tx=0x7f1ecc00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:21.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc018030 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ecc00cd30 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed41e9180 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc003bb0 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41e9740 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1ecc01b070 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.949+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f1ecc01d930 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.950+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed4075650 con 0x7f1ed41573a0 2024-05-09T07:13:21.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:21.952+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1ecc010040 con 0x7f1ed41573a0 2024-05-09T07:13:22.027 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.025+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "348cc74d-e143-48f9-a79f-225bd020ac74", "id": 1} v 0) -- 0x7f1ed40c62a0 con 0x7f1ed41573a0 2024-05-09T07:13:22.036 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.034+0000 7f1ed0ff9640 1 -- 172.21.15.188:0/2462266866 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "348cc74d-e143-48f9-a79f-225bd020ac74", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f1ecc01dcb0 con 0x7f1ed41573a0 2024-05-09T07:13:22.036 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 msgr2=0x7f1ed414b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed414b3d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00efc0 tx=0x7f1ecc00c490 comp rx=0 tx=0).stop 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 shutdown_connections 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1ed41573a0 0x7f1ed414b3d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ed40c4690 0x7f1ed414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 --2- 172.21.15.188:0/2462266866 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1ed406fac0 0x7f1ed4155df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 >> 172.21.15.188:0/2462266866 conn(0x7f1ed4072a80 msgr2=0x7f1ed40ac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 shutdown_connections 2024-05-09T07:13:22.037 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.035+0000 7f1eda6ba640 1 -- 172.21.15.188:0/2462266866 wait complete. 2024-05-09T07:13:22.048 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new a146b295-ee51-4e60-9ec9-aa7880de8049 2 2024-05-09T07:13:22.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.145+0000 7f69e5163640 1 Processor -- start 2024-05-09T07:13:22.147 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69e5163640 1 -- start start 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69e5163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e0073150 0x7f69e006f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69e5163640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69e00c60d0 0x7f69e00c7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69e5163640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e00c76b0 con 0x7f69e0073640 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69e5163640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f69e00c2650 con 0x7f69e00c60d0 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69e5163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69e0152590 con 0x7f69e0073150 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69ded9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e0073150 0x7f69e006f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69df59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f69e00c60d0 0x7f69e00c7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69de59c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f69e0073640 0x7f69e00c59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:51168/0 (socket says 172.21.15.188:51168) 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.146+0000 7f69de59c640 1 -- 172.21.15.188:0/3630486117 learned_addr learned my addr 172.21.15.188:0/3630486117 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.147+0000 7f69df59e640 1 -- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e0073150 msgr2=0x7f69e006f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:22.150 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.147+0000 7f69df59e640 1 --2- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e0073150 0x7f69e006f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.147+0000 7f69df59e640 1 -- 172.21.15.188:0/3630486117 >> v1:172.21.15.188:6789/0 conn(0x7f69e0073640 legacy=0x7f69e00c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.147+0000 7f69df59e640 1 -- 172.21.15.188:0/3630486117 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e0152870 con 0x7f69e00c60d0 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.147+0000 7f69ded9d640 1 --2- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e0073150 0x7f69e006f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.147+0000 7f69df59e640 1 --2- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00c60d0 0x7f69e00c7a50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009d90 tx=0x7f69d802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e76891056696df2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69ddd9b640 1 -- 172.21.15.188:0/3630486117 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8007740 con 0x7f69e00c60d0 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69ddd9b640 1 -- 172.21.15.188:0/3630486117 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d80078e0 con 0x7f69e00c60d0 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69ddd9b640 1 -- 172.21.15.188:0/3630486117 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8007cb0 con 0x7f69e00c60d0 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 -- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00c60d0 msgr2=0x7f69e00c7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 --2- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00c60d0 0x7f69e00c7a50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69d8009d90 tx=0x7f69d802a560 comp rx=0 tx=0).stop 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 -- 172.21.15.188:0/3630486117 shutdown_connections 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 --2- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e00c60d0 0x7f69e00c7a50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 --2- 172.21.15.188:0/3630486117 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e0073150 0x7f69e006f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 -- 172.21.15.188:0/3630486117 >> 172.21.15.188:0/3630486117 conn(0x7f69e006de80 msgr2=0x7f69e006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.151 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 -- 172.21.15.188:0/3630486117 shutdown_connections 2024-05-09T07:13:22.152 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.148+0000 7f69e5163640 1 -- 172.21.15.188:0/3630486117 wait complete. 2024-05-09T07:13:22.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 Processor -- start 2024-05-09T07:13:22.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 -- start start 2024-05-09T07:13:22.154 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 0x7f69e01d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e0073640 0x7f69e00abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00c60d0 0x7f69e01d95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0153070 con 0x7f69e0073640 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69e0152280 con 0x7f69e0073150 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69e5163640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69e01d8980 con 0x7f69e00c60d0 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69ded9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 0x7f69e01d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69ded9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 0x7f69e01d8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53516/0 (socket says 172.21.15.188:53516) 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69ded9d640 1 -- 172.21.15.188:0/3970099315 learned_addr learned my addr 172.21.15.188:0/3970099315 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.150+0000 7f69df59e640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00c60d0 0x7f69e01d95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69de59c640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e0073640 0x7f69e00abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69ded9d640 1 -- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00c60d0 msgr2=0x7f69e01d95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69ded9d640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00c60d0 0x7f69e01d95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.155 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69ded9d640 1 -- 172.21.15.188:0/3970099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e0073640 msgr2=0x7f69e00abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69ded9d640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e0073640 0x7f69e00abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69ded9d640 1 -- 172.21.15.188:0/3970099315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e01d9350 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69ded9d640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 0x7f69e01d8e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f69d0010f40 tx=0x7f69d000e7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d001e070 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d0011860 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e00a83d0 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d0011c10 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e01d8610 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f69d0021070 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.151+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f69d0023d30 con 0x7f69e0073150 2024-05-09T07:13:22.156 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.152+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e00a88f0 con 0x7f69e0073150 2024-05-09T07:13:22.157 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.154+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f69d0011db0 con 0x7f69e0073150 2024-05-09T07:13:22.228 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.227+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a146b295-ee51-4e60-9ec9-aa7880de8049", "id": 2} v 0) -- 0x7f69e01d8400 con 0x7f69e0073150 2024-05-09T07:13:22.234 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.232+0000 7f69d77fe640 1 -- 172.21.15.188:0/3970099315 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a146b295-ee51-4e60-9ec9-aa7880de8049", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f69d0023510 con 0x7f69e0073150 2024-05-09T07:13:22.234 INFO:teuthology.orchestra.run.smithi188.stdout:2 2024-05-09T07:13:22.235 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 msgr2=0x7f69e01d8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.235 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 0x7f69e01d8e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f69d0010f40 tx=0x7f69d000e7f0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.235 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 shutdown_connections 2024-05-09T07:13:22.235 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69e00c60d0 0x7f69e01d95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.235 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69e0073640 0x7f69e00abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.235 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 --2- 172.21.15.188:0/3970099315 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69e0073150 0x7f69e01d8e10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 >> 172.21.15.188:0/3970099315 conn(0x7f69e006de80 msgr2=0x7f69e0150280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 shutdown_connections 2024-05-09T07:13:22.236 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.233+0000 7f69e5163640 1 -- 172.21.15.188:0/3970099315 wait complete. 2024-05-09T07:13:22.245 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 1ff30033-a2e4-4b39-962b-c6dab56a96b8 3 2024-05-09T07:13:22.345 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.344+0000 7f53f4e31640 1 Processor -- start 2024-05-09T07:13:22.346 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.344+0000 7f53f4e31640 1 -- start start 2024-05-09T07:13:22.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53f4e31640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f00c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53f4e31640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f53f00c6fb0 0x7f53f015fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53f4e31640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f0152f40 con 0x7f53f0152b60 2024-05-09T07:13:22.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53f4e31640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f53f006b430 con 0x7f53f00c6fb0 2024-05-09T07:13:22.347 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53f4e31640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f53f01598f0 con 0x7f53f00c1900 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53edd9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f00c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53ee59c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f53f0152b60 0x7f53f006b840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:51182/0 (socket says 172.21.15.188:51182) 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53ee59c640 1 -- 172.21.15.188:0/2420517061 learned_addr learned my addr 172.21.15.188:0/2420517061 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53eed9d640 1 --2- 172.21.15.188:0/2420517061 >> v2:172.21.15.160:3300/0 conn(0x7f53f00c6fb0 0x7f53f015fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53edd9b640 1 -- 172.21.15.188:0/2420517061 >> v2:172.21.15.160:3300/0 conn(0x7f53f00c6fb0 msgr2=0x7f53f015fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53edd9b640 1 --2- 172.21.15.188:0/2420517061 >> v2:172.21.15.160:3300/0 conn(0x7f53f00c6fb0 0x7f53f015fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53ed59a640 1 -- 172.21.15.188:0/2420517061 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3650490377 0 0) 0x7f53e0001110 con 0x7f53f0152b60 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53edd9b640 1 -- 172.21.15.188:0/2420517061 >> v1:172.21.15.188:6789/0 conn(0x7f53f0152b60 legacy=0x7f53f006b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.345+0000 7f53edd9b640 1 -- 172.21.15.188:0/2420517061 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0158b30 con 0x7f53f00c1900 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.346+0000 7f53edd9b640 1 --2- 172.21.15.188:0/2420517061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f00c6a70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53e40097a0 tx=0x7f53e402a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94f5e3bce2d0eae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.346+0000 7f53ed59a640 1 -- 172.21.15.188:0/2420517061 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4037070 con 0x7f53f00c1900 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.346+0000 7f53ed59a640 1 -- 172.21.15.188:0/2420517061 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e402adb0 con 0x7f53f00c1900 2024-05-09T07:13:22.348 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.346+0000 7f53ed59a640 1 -- 172.21.15.188:0/2420517061 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e4032660 con 0x7f53f00c1900 2024-05-09T07:13:22.353 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 -- 172.21.15.188:0/2420517061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 msgr2=0x7f53f00c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 --2- 172.21.15.188:0/2420517061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f00c6a70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f53e40097a0 tx=0x7f53e402a060 comp rx=0 tx=0).stop 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 -- 172.21.15.188:0/2420517061 shutdown_connections 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 --2- 172.21.15.188:0/2420517061 >> v2:172.21.15.160:3300/0 conn(0x7f53f00c6fb0 0x7f53f015fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 --2- 172.21.15.188:0/2420517061 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f00c6a70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 -- 172.21.15.188:0/2420517061 >> 172.21.15.188:0/2420517061 conn(0x7f53f006de80 msgr2=0x7f53f006c270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 -- 172.21.15.188:0/2420517061 shutdown_connections 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.347+0000 7f53f4e31640 1 -- 172.21.15.188:0/2420517061 wait complete. 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.348+0000 7f53f4e31640 1 Processor -- start 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.348+0000 7f53f4e31640 1 -- start start 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53f4e31640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f0154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53f4e31640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53f00c6fb0 0x7f53f00c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53f4e31640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53f0152b60 0x7f53f01ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.354 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53f4e31640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f53f00a74c0 con 0x7f53f0152b60 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53f4e31640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f53f01590f0 con 0x7f53f00c6fb0 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53f4e31640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f53f00c1ea0 con 0x7f53f00c1900 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f0154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53edd9b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53f00c6fb0 0x7f53f00c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f0154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52344/0 (socket says 172.21.15.188:52344) 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 -- 172.21.15.188:0/428265530 learned_addr learned my addr 172.21.15.188:0/428265530 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53eed9d640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53f0152b60 0x7f53f01ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 -- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53f00c6fb0 msgr2=0x7f53f00c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53f00c6fb0 0x7f53f00c37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 -- 172.21.15.188:0/428265530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53f0152b60 msgr2=0x7f53f01ee9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.355 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53f0152b60 0x7f53f01ee9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.356 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 -- 172.21.15.188:0/428265530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f0153580 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53ee59c640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f0154300 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c780 tx=0x7f53e000cc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.349+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e0018070 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.350+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53e00072d0 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.350+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53e00047e0 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.350+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f0153860 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.350+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f00c5130 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.350+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53e001b070 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.350+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f53e0012400 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.351+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f0153a30 con 0x7f53f00c1900 2024-05-09T07:13:22.357 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.353+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f53f0152f40 con 0x7f53f00c1900 2024-05-09T07:13:22.387 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.385+0000 7f5cb480d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T07:13:22.427 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.426+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1ff30033-a2e4-4b39-962b-c6dab56a96b8", "id": 3} v 0) -- 0x7f53f0075650 con 0x7f53f00c1900 2024-05-09T07:13:22.432 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.431+0000 7f53deffd640 1 -- 172.21.15.188:0/428265530 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1ff30033-a2e4-4b39-962b-c6dab56a96b8", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f53e000aaa0 con 0x7f53f00c1900 2024-05-09T07:13:22.433 INFO:teuthology.orchestra.run.smithi188.stdout:3 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 msgr2=0x7f53f0154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f0154300 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f53e000c780 tx=0x7f53e000cc40 comp rx=0 tx=0).stop 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 shutdown_connections 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f53f00c6fb0 0x7f53f00c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f53f0152b60 0x7f53f01ee9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 --2- 172.21.15.188:0/428265530 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f53f00c1900 0x7f53f0154300 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 >> 172.21.15.188:0/428265530 conn(0x7f53f006de80 msgr2=0x7f53f0151490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 shutdown_connections 2024-05-09T07:13:22.434 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.432+0000 7f53f4e31640 1 -- 172.21.15.188:0/428265530 wait complete. 2024-05-09T07:13:22.440 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.439+0000 7f0b0ec12040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T07:13:22.444 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 11b9a811-6789-4a48-aab8-e9066ea7440f 4 2024-05-09T07:13:22.475 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.473+0000 7f5cb480d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T07:13:22.517 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.516+0000 7f5cb480d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T07:13:22.533 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.531+0000 7f0b0ec12040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T07:13:22.543 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.542+0000 7f2a0cf63640 1 Processor -- start 2024-05-09T07:13:22.544 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.542+0000 7f2a0cf63640 1 -- start start 2024-05-09T07:13:22.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a0cf63640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2a080c1fb0 0x7f2a08155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a0cf63640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a080c07d0 0x7f2a080c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a0cf63640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a08155440 con 0x7f2a08073490 2024-05-09T07:13:22.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a0cf63640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f2a0806b870 con 0x7f2a080c1fb0 2024-05-09T07:13:22.545 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a0cf63640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2a08155a20 con 0x7f2a080c07d0 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a06d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a080c07d0 0x7f2a080c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a06d9d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2a080c07d0 0x7f2a080c0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53528/0 (socket says 172.21.15.188:53528) 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2691710493 learned_addr learned my addr 172.21.15.188:0/2691710493 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.543+0000 7f2a05d9b640 1 --2- 172.21.15.188:0/2691710493 >> v2:172.21.15.160:3300/0 conn(0x7f2a080c1fb0 0x7f2a08155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.544+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2691710493 >> v2:172.21.15.160:3300/0 conn(0x7f2a080c1fb0 msgr2=0x7f2a08155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.544+0000 7f2a06d9d640 1 --2- 172.21.15.188:0/2691710493 >> v2:172.21.15.160:3300/0 conn(0x7f2a080c1fb0 0x7f2a08155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.544+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2691710493 >> v1:172.21.15.160:6789/0 conn(0x7f2a08073490 legacy=0x7f2a0806b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.546 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.544+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2691710493 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a080c1400 con 0x7f2a080c07d0 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.545+0000 7f2a06d9d640 1 --2- 172.21.15.188:0/2691710493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c07d0 0x7f2a080c0c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a0806d020 tx=0x7f2a0002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c2e2e6f8721563 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.545+0000 7f2a0559a640 1 -- 172.21.15.188:0/2691710493 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00036070 con 0x7f2a080c07d0 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.545+0000 7f2a0559a640 1 -- 172.21.15.188:0/2691710493 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a00031600 con 0x7f2a080c07d0 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.545+0000 7f2a0559a640 1 -- 172.21.15.188:0/2691710493 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a000319d0 con 0x7f2a080c07d0 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2691710493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c07d0 msgr2=0x7f2a080c0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2691710493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c07d0 0x7f2a080c0c80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2a0806d020 tx=0x7f2a0002a6a0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2691710493 shutdown_connections 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2691710493 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c07d0 0x7f2a080c0c80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2691710493 >> v2:172.21.15.160:3300/0 conn(0x7f2a080c1fb0 0x7f2a08155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2691710493 >> 172.21.15.188:0/2691710493 conn(0x7f2a080c3ea0 msgr2=0x7f2a080c42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2691710493 shutdown_connections 2024-05-09T07:13:22.548 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.546+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2691710493 wait complete. 2024-05-09T07:13:22.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.548+0000 7f2a0cf63640 1 Processor -- start 2024-05-09T07:13:22.549 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.548+0000 7f2a0cf63640 1 -- start start 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0cf63640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a08073490 0x7f2a08147d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0cf63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a080c07d0 0x7f2a08146ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0cf63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 0x7f2a08146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0cf63640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2a08067560 con 0x7f2a08073490 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0cf63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2a081556b0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0cf63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2a080c19c0 con 0x7f2a080c07d0 2024-05-09T07:13:22.552 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a06d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 0x7f2a08146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a05d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a080c07d0 0x7f2a08146ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a06d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 0x7f2a08146ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53538/0 (socket says 172.21.15.188:53538) 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2211434588 learned_addr learned my addr 172.21.15.188:0/2211434588 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a0659c640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a08073490 0x7f2a08147d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a080c07d0 msgr2=0x7f2a08146ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.549+0000 7f2a06d9d640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a080c07d0 0x7f2a08146ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2211434588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a08073490 msgr2=0x7f2a08147d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a06d9d640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a08073490 0x7f2a08147d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a06d9d640 1 -- 172.21.15.188:0/2211434588 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a081482a0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a0659c640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a08073490 0x7f2a08147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a05d9b640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a080c07d0 0x7f2a08146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a06d9d640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 0x7f2a08146ff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009f40 tx=0x7f2a00004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00036050 con 0x7f2a080c1fb0 2024-05-09T07:13:22.553 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a00042070 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a081ebef0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a00009510 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a080aa5c0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2a00007bf0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.550+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f2a0003d450 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.551+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a080aa7c0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.554 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.553+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f2a080aa7c0 con 0x7f2a080c1fb0 2024-05-09T07:13:22.580 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.579+0000 7f0b0ec12040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T07:13:22.628 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.626+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "11b9a811-6789-4a48-aab8-e9066ea7440f", "id": 4} v 0) -- 0x7f2a08075650 con 0x7f2a080c1fb0 2024-05-09T07:13:22.633 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.632+0000 7f29feffd640 1 -- 172.21.15.188:0/2211434588 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "11b9a811-6789-4a48-aab8-e9066ea7440f", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f2a00047460 con 0x7f2a080c1fb0 2024-05-09T07:13:22.633 INFO:teuthology.orchestra.run.smithi188.stdout:4 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 msgr2=0x7f2a08146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 0x7f2a08146ff0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2a00009f40 tx=0x7f2a00004a40 comp rx=0 tx=0).stop 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 shutdown_connections 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2a080c1fb0 0x7f2a08146ff0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2a080c07d0 0x7f2a08146ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 --2- 172.21.15.188:0/2211434588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a08073490 0x7f2a08147d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 >> 172.21.15.188:0/2211434588 conn(0x7f2a080c3ea0 msgr2=0x7f2a08154100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 shutdown_connections 2024-05-09T07:13:22.635 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.633+0000 7f2a0cf63640 1 -- 172.21.15.188:0/2211434588 wait complete. 2024-05-09T07:13:22.645 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new a4fc7ad0-cf97-4459-84e5-f63ad1e45658 5 2024-05-09T07:13:22.685 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.683+0000 7f5cb480d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T07:13:22.737 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.736+0000 7f5cb480d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T07:13:22.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.744+0000 7f0671695640 1 Processor -- start 2024-05-09T07:13:22.745 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.744+0000 7f0671695640 1 -- start start 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0671695640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f066c06fac0 0x7f066c0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0671695640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066c0c7620 con 0x7f066c1573a0 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0671695640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f066c0c4950 con 0x7f066c0c44c0 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0671695640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f066c0714e0 con 0x7f066c06fac0 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f066b7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f066c0c44c0 0x7f066c15dc70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:51194/0 (socket says 172.21.15.188:51194) 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f066b7fe640 1 -- 172.21.15.188:0/862555374 learned_addr learned my addr 172.21.15.188:0/862555374 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f066a7fc640 1 --2- 172.21.15.188:0/862555374 >> v2:172.21.15.160:3300/0 conn(0x7f066c06fac0 0x7f066c0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0669ffb640 1 -- 172.21.15.188:0/862555374 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3813207254 0 0) 0x7f066c0c4950 con 0x7f066c0c44c0 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0669ffb640 1 -- 172.21.15.188:0/862555374 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0658003520 con 0x7f066c0c44c0 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f066a7fc640 1 -- 172.21.15.188:0/862555374 >> v1:172.21.15.188:6789/0 conn(0x7f066c0c44c0 legacy=0x7f066c15dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f066a7fc640 1 -- 172.21.15.188:0/862555374 >> v1:172.21.15.160:6789/0 conn(0x7f066c1573a0 legacy=0x7f066c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.747 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f066a7fc640 1 -- 172.21.15.188:0/862555374 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c071200 con 0x7f066c06fac0 2024-05-09T07:13:22.748 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.745+0000 7f0669ffb640 1 -- 172.21.15.188:0/862555374 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 268635792 0 0) 0x7f066c0c7620 con 0x7f066c1573a0 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.746+0000 7f066a7fc640 1 --2- 172.21.15.188:0/862555374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c0c3f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f065c0097a0 tx=0x7f065c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=500d72cd85a3b09d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.746+0000 7f0669ffb640 1 -- 172.21.15.188:0/862555374 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c037070 con 0x7f066c06fac0 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.746+0000 7f0669ffb640 1 -- 172.21.15.188:0/862555374 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c032640 con 0x7f066c06fac0 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.746+0000 7f0669ffb640 1 -- 172.21.15.188:0/862555374 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c032a10 con 0x7f066c06fac0 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 -- 172.21.15.188:0/862555374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 msgr2=0x7f066c0c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 --2- 172.21.15.188:0/862555374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c0c3f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f065c0097a0 tx=0x7f065c02a2c0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 -- 172.21.15.188:0/862555374 shutdown_connections 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 --2- 172.21.15.188:0/862555374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c0c3f80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 -- 172.21.15.188:0/862555374 >> 172.21.15.188:0/862555374 conn(0x7f066c072a80 msgr2=0x7f066c072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.749 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 -- 172.21.15.188:0/862555374 shutdown_connections 2024-05-09T07:13:22.750 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.747+0000 7f0671695640 1 -- 172.21.15.188:0/862555374 wait complete. 2024-05-09T07:13:22.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.748+0000 7f0671695640 1 Processor -- start 2024-05-09T07:13:22.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.748+0000 7f0671695640 1 -- start start 2024-05-09T07:13:22.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f0671695640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c06de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f0671695640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 0x7f066c1ee840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f0671695640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f066c1573a0 0x7f066c1ea950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.752 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f0671695640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f066c0c3af0 con 0x7f066c06fac0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f0671695640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f066c0c78c0 con 0x7f066c0c44c0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f0671695640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f066c0abc20 con 0x7f066c1573a0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 0x7f066c1ee840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 0x7f066c1ee840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53540/0 (socket says 172.21.15.188:53540) 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 -- 172.21.15.188:0/4276992328 learned_addr learned my addr 172.21.15.188:0/4276992328 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066b7fe640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f066c1573a0 0x7f066c1ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066affd640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c06de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 -- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f066c1573a0 msgr2=0x7f066c1ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f066c1573a0 0x7f066c1ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 -- 172.21.15.188:0/4276992328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 msgr2=0x7f066c06de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c06de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 -- 172.21.15.188:0/4276992328 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f066c0a8f40 con 0x7f066c0c44c0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f066a7fc640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 0x7f066c1ee840 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f065c0098c0 tx=0x7f065c0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.753 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c041020 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f065c0306f0 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.749+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f065c037040 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.750+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f066c0ad0d0 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.750+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f066c0a91e0 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.750+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f065c044020 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.750+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f065c03dba0 con 0x7f066c0c44c0 2024-05-09T07:13:22.754 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.750+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f066c075650 con 0x7f066c0c44c0 2024-05-09T07:13:22.755 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.754+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f065c04b030 con 0x7f066c0c44c0 2024-05-09T07:13:22.757 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.756+0000 7f0b0ec12040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T07:13:22.790 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.789+0000 7f5cb480d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T07:13:22.806 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.805+0000 7f0b0ec12040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T07:13:22.829 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.827+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "a4fc7ad0-cf97-4459-84e5-f63ad1e45658", "id": 5} v 0) -- 0x7f066c0c4950 con 0x7f066c0c44c0 2024-05-09T07:13:22.834 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.833+0000 7f06637fe640 1 -- 172.21.15.188:0/4276992328 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a4fc7ad0-cf97-4459-84e5-f63ad1e45658", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f065c03d420 con 0x7f066c0c44c0 2024-05-09T07:13:22.835 INFO:teuthology.orchestra.run.smithi188.stdout:5 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 msgr2=0x7f066c1ee840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 0x7f066c1ee840 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f065c0098c0 tx=0x7f065c0046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 shutdown_connections 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f066c0c44c0 0x7f066c1ee840 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f066c1573a0 0x7f066c1ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 --2- 172.21.15.188:0/4276992328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f066c06fac0 0x7f066c06de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 >> 172.21.15.188:0/4276992328 conn(0x7f066c072a80 msgr2=0x7f066c0ad5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 shutdown_connections 2024-05-09T07:13:22.836 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.834+0000 7f0671695640 1 -- 172.21.15.188:0/4276992328 wait complete. 2024-05-09T07:13:22.846 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 894aeb3c-d190-4d55-85f0-908ba67e54ca 6 2024-05-09T07:13:22.863 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.862+0000 7f0b0ec12040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T07:13:22.903 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:22.902+0000 7f5cb480d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T07:13:22.946 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.944+0000 7f06a7d5a640 1 Processor -- start 2024-05-09T07:13:22.946 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.945+0000 7f06a7d5a640 1 -- start start 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.945+0000 7f06a7d5a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f06a0152a30 0x7f06a0070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.945+0000 7f06a7d5a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06a00c8aa0 0x7f06a00c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.945+0000 7f06a7d5a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06a0070690 con 0x7f06a00c2a00 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.945+0000 7f06a7d5a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f06a00c82a0 con 0x7f06a0152a30 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.945+0000 7f06a7d5a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f06a0071360 con 0x7f06a00c8aa0 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06a00c8aa0 0x7f06a00c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f06a00c8aa0 0x7f06a00c8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53554/0 (socket says 172.21.15.188:53554) 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 -- 172.21.15.188:0/2640781460 learned_addr learned my addr 172.21.15.188:0/2640781460 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a5af6640 1 --2- 172.21.15.188:0/2640781460 >> v2:172.21.15.160:3300/0 conn(0x7f06a0152a30 0x7f06a0070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 -- 172.21.15.188:0/2640781460 >> v2:172.21.15.160:3300/0 conn(0x7f06a0152a30 msgr2=0x7f06a0070e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 --2- 172.21.15.188:0/2640781460 >> v2:172.21.15.160:3300/0 conn(0x7f06a0152a30 0x7f06a0070e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 -- 172.21.15.188:0/2640781460 >> v1:172.21.15.160:6789/0 conn(0x7f06a00c2a00 legacy=0x7f06a00c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.948 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a4af4640 1 -- 172.21.15.188:0/2640781460 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4147372762 0 0) 0x7f06a0070690 con 0x7f06a00c2a00 2024-05-09T07:13:22.949 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a62f7640 1 -- 172.21.15.188:0/2640781460 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a00a7d40 con 0x7f06a00c8aa0 2024-05-09T07:13:22.949 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.946+0000 7f06a5af6640 1 --2- 172.21.15.188:0/2640781460 >> v2:172.21.15.160:3300/0 conn(0x7f06a0152a30 0x7f06a0070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.947+0000 7f06a62f7640 1 --2- 172.21.15.188:0/2640781460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c8aa0 0x7f06a00c8ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06a006e450 tx=0x7f069802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8fedb5233e69a44b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.947+0000 7f06a4af4640 1 -- 172.21.15.188:0/2640781460 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0698036070 con 0x7f06a00c8aa0 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.947+0000 7f06a4af4640 1 -- 172.21.15.188:0/2640781460 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0698031600 con 0x7f06a00c8aa0 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.947+0000 7f06a4af4640 1 -- 172.21.15.188:0/2640781460 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06980319d0 con 0x7f06a00c8aa0 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 -- 172.21.15.188:0/2640781460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c8aa0 msgr2=0x7f06a00c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/2640781460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c8aa0 0x7f06a00c8ed0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f06a006e450 tx=0x7f069802a730 comp rx=0 tx=0).stop 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 -- 172.21.15.188:0/2640781460 shutdown_connections 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/2640781460 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c8aa0 0x7f06a00c8ed0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.950 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/2640781460 >> v2:172.21.15.160:3300/0 conn(0x7f06a0152a30 0x7f06a0070e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 -- 172.21.15.188:0/2640781460 >> 172.21.15.188:0/2640781460 conn(0x7f06a0072a80 msgr2=0x7f06a0072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:22.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 -- 172.21.15.188:0/2640781460 shutdown_connections 2024-05-09T07:13:22.951 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.948+0000 7f06a7d5a640 1 -- 172.21.15.188:0/2640781460 wait complete. 2024-05-09T07:13:22.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.949+0000 7f06a7d5a640 1 Processor -- start 2024-05-09T07:13:22.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.949+0000 7f06a7d5a640 1 -- start start 2024-05-09T07:13:22.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a7d5a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 0x7f06a014f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a7d5a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a00c8aa0 0x7f06a00accb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a7d5a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a0152a30 0x7f06a0149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a7d5a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f06a0070900 con 0x7f06a00c8aa0 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a7d5a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f06a00a8300 con 0x7f06a00c2a00 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a7d5a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f06a0152e10 con 0x7f06a0152a30 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 0x7f06a014f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 0x7f06a014f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53564/0 (socket says 172.21.15.188:53564) 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 -- 172.21.15.188:0/1327814291 learned_addr learned my addr 172.21.15.188:0/1327814291 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a62f7640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a0152a30 0x7f06a0149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a52f5640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a00c8aa0 0x7f06a00accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 -- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a0152a30 msgr2=0x7f06a0149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a0152a30 0x7f06a0149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 -- 172.21.15.188:0/1327814291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a00c8aa0 msgr2=0x7f06a00accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:22.954 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a00c8aa0 0x7f06a00accb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 -- 172.21.15.188:0/1327814291 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06a01ebe90 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a52f5640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a00c8aa0 0x7f06a00accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a62f7640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a0152a30 0x7f06a0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.950+0000 7f06a5af6640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 0x7f06a014f770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f069c00ebe0 tx=0x7f069c00c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c017070 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f069c00cc30 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06a00c46f0 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069c013470 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06a00ad1f0 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f069c01a070 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.951+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f069c006020 con 0x7f06a00c2a00 2024-05-09T07:13:22.955 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.952+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06a0075650 con 0x7f06a00c2a00 2024-05-09T07:13:22.956 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:22.954+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f069c0139d0 con 0x7f06a00c2a00 2024-05-09T07:13:22.981 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:22.980+0000 7f0b0ec12040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T07:13:23.026 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.025+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "894aeb3c-d190-4d55-85f0-908ba67e54ca", "id": 6} v 0) -- 0x7f06a00c1fa0 con 0x7f06a00c2a00 2024-05-09T07:13:23.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.030+0000 7f06967fc640 1 -- 172.21.15.188:0/1327814291 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "894aeb3c-d190-4d55-85f0-908ba67e54ca", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f069c013bb0 con 0x7f06a00c2a00 2024-05-09T07:13:23.031 INFO:teuthology.orchestra.run.smithi188.stdout:6 2024-05-09T07:13:23.032 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 msgr2=0x7f06a014f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 0x7f06a014f770 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f069c00ebe0 tx=0x7f069c00c520 comp rx=0 tx=0).stop 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 shutdown_connections 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f06a0152a30 0x7f06a0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f06a00c8aa0 0x7f06a00accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 --2- 172.21.15.188:0/1327814291 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f06a00c2a00 0x7f06a014f770 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 >> 172.21.15.188:0/1327814291 conn(0x7f06a0072a80 msgr2=0x7f06a0155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 shutdown_connections 2024-05-09T07:13:23.033 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.031+0000 7f06a7d5a640 1 -- 172.21.15.188:0/1327814291 wait complete. 2024-05-09T07:13:23.043 DEBUG:teuthology.orchestra.run.smithi188:> sudo ceph --cluster ceph osd new 4671e883-f5b6-46de-9280-7b1297eca4de 7 2024-05-09T07:13:23.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.182+0000 7f7adb3b7640 1 Processor -- start 2024-05-09T07:13:23.184 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.182+0000 7f7adb3b7640 1 -- start start 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7adb3b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 0x7f7ad40a8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7adb3b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ad40c86f0 0x7f7ad40acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7adb3b7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ad4150dc0 con 0x7f7ad4156850 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7adb3b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7ad4153680 con 0x7f7ad40c86f0 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7adb3b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad4153960 con 0x7f7ad40a7da0 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7ad9153640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 0x7f7ad40a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7ad9954640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7ad4156850 0x7f7ad40ad310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.188:51202/0 (socket says 172.21.15.188:51202) 2024-05-09T07:13:23.187 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7ad8952640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7ad40c86f0 0x7f7ad40acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7ad9954640 1 -- 172.21.15.188:0/1153948178 learned_addr learned my addr 172.21.15.188:0/1153948178 (peer_addr_for_me v1:172.21.15.188:0/0) 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.183+0000 7f7ad9153640 1 --2- 172.21.15.188:0/1153948178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 0x7f7ad40a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.188:52376/0 (socket says 172.21.15.188:52376) 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.184+0000 7f7ad9153640 1 -- 172.21.15.188:0/1153948178 >> v2:172.21.15.188:3300/0 conn(0x7f7ad40c86f0 msgr2=0x7f7ad40acdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.184+0000 7f7ad9153640 1 --2- 172.21.15.188:0/1153948178 >> v2:172.21.15.188:3300/0 conn(0x7f7ad40c86f0 0x7f7ad40acdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.184+0000 7f7ad9153640 1 -- 172.21.15.188:0/1153948178 >> v1:172.21.15.188:6789/0 conn(0x7f7ad4156850 legacy=0x7f7ad40ad310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.184+0000 7f7ad9153640 1 -- 172.21.15.188:0/1153948178 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad40a86c0 con 0x7f7ad40a7da0 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.184+0000 7f7ad8952640 1 --2- 172.21.15.188:0/1153948178 >> v2:172.21.15.188:3300/0 conn(0x7f7ad40c86f0 0x7f7ad40acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.184+0000 7f7ad9153640 1 --2- 172.21.15.188:0/1153948178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 0x7f7ad40a8180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40097a0 tx=0x7f7ac402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4f3f1e4834e8ff1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7acbfff640 1 -- 172.21.15.188:0/1153948178 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4037070 con 0x7f7ad40a7da0 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7acbfff640 1 -- 172.21.15.188:0/1153948178 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac4032440 con 0x7f7ad40a7da0 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7acbfff640 1 -- 172.21.15.188:0/1153948178 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4032810 con 0x7f7ad40a7da0 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 -- 172.21.15.188:0/1153948178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 msgr2=0x7f7ad40a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/1153948178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 0x7f7ad40a8180 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40097a0 tx=0x7f7ac402a180 comp rx=0 tx=0).stop 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 -- 172.21.15.188:0/1153948178 shutdown_connections 2024-05-09T07:13:23.188 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/1153948178 >> v2:172.21.15.188:3300/0 conn(0x7f7ad40c86f0 0x7f7ad40acdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/1153948178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad40a7da0 0x7f7ad40a8180 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 -- 172.21.15.188:0/1153948178 >> 172.21.15.188:0/1153948178 conn(0x7f7ad40c8b70 msgr2=0x7f7ad40c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:23.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 -- 172.21.15.188:0/1153948178 shutdown_connections 2024-05-09T07:13:23.189 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.185+0000 7f7adb3b7640 1 -- 172.21.15.188:0/1153948178 wait complete. 2024-05-09T07:13:23.191 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.186+0000 7f7adb3b7640 1 Processor -- start 2024-05-09T07:13:23.191 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.186+0000 7f7adb3b7640 1 -- start start 2024-05-09T07:13:23.191 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7adb3b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 0x7f7ad406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7adb3b7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad40c86f0 0x7f7ad40c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7adb3b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad4156850 0x7f7ad41eb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7adb3b7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad4153330 con 0x7f7ad40c86f0 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7adb3b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad4153f20 con 0x7f7ad40a7da0 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7adb3b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad40ada20 con 0x7f7ad4156850 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 0x7f7ad406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9954640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad4156850 0x7f7ad41eb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 0x7f7ad406c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.188:53586/0 (socket says 172.21.15.188:53586) 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 -- 172.21.15.188:0/3221186217 learned_addr learned my addr 172.21.15.188:0/3221186217 (peer_addr_for_me v2:172.21.15.188:0/0) 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad8952640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad40c86f0 0x7f7ad40c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 -- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad4156850 msgr2=0x7f7ad41eb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad4156850 0x7f7ad41eb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 -- 172.21.15.188:0/3221186217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad40c86f0 msgr2=0x7f7ad40c1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.192 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad40c86f0 0x7f7ad40c1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.187+0000 7f7ad9153640 1 -- 172.21.15.188:0/3221186217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad406c310 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ad9954640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad4156850 0x7f7ad41eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ad8952640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad40c86f0 0x7f7ad40c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ad9153640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 0x7f7ad406c7f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40098c0 tx=0x7f7ac40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4037040 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ac4030500 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad41547d0 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40308d0 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad41516c0 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7ac4044020 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.188+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f7ac403bbb0 con 0x7f7ad40a7da0 2024-05-09T07:13:23.193 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.189+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad406c4e0 con 0x7f7ad40a7da0 2024-05-09T07:13:23.194 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.191+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f7ac403b430 con 0x7f7ad40a7da0 2024-05-09T07:13:23.265 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.264+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4671e883-f5b6-46de-9280-7b1297eca4de", "id": 7} v 0) -- 0x7f7ad4075650 con 0x7f7ad40a7da0 2024-05-09T07:13:23.272 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.270+0000 7f7ac9ffb640 1 -- 172.21.15.188:0/3221186217 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4671e883-f5b6-46de-9280-7b1297eca4de", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f7ac403b610 con 0x7f7ad40a7da0 2024-05-09T07:13:23.272 INFO:teuthology.orchestra.run.smithi188.stdout:7 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 msgr2=0x7f7ad406c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 0x7f7ad406c7f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40098c0 tx=0x7f7ac40046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 shutdown_connections 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ad4156850 0x7f7ad41eb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ad40c86f0 0x7f7ad40c1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 --2- 172.21.15.188:0/3221186217 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ad40a7da0 0x7f7ad406c7f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 >> 172.21.15.188:0/3221186217 conn(0x7f7ad40c8b70 msgr2=0x7f7ad40acfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 shutdown_connections 2024-05-09T07:13:23.273 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:13:23.271+0000 7f7adb3b7640 1 -- 172.21.15.188:0/3221186217 wait complete. 2024-05-09T07:13:23.283 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-09T07:13:23.283 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-09T07:13:23.286 INFO:tasks.ceph.osd.0:Started 2024-05-09T07:13:23.286 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-09T07:13:23.286 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-09T07:13:23.287 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:23.284+0000 7f5cb480d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T07:13:23.288 INFO:tasks.ceph.osd.1:Started 2024-05-09T07:13:23.288 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-09T07:13:23.289 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-09T07:13:23.292 INFO:tasks.ceph.osd.2:Started 2024-05-09T07:13:23.292 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-09T07:13:23.293 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-09T07:13:23.295 INFO:tasks.ceph.osd.3:Started 2024-05-09T07:13:23.296 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-09T07:13:23.296 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-09T07:13:23.298 INFO:tasks.ceph.osd.4:Started 2024-05-09T07:13:23.298 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-09T07:13:23.299 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-09T07:13:23.301 INFO:tasks.ceph.osd.5:Started 2024-05-09T07:13:23.301 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-09T07:13:23.301 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-09T07:13:23.303 INFO:tasks.ceph.osd.6:Started 2024-05-09T07:13:23.303 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-09T07:13:23.303 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-09T07:13:23.305 INFO:tasks.ceph.osd.7:Started 2024-05-09T07:13:23.306 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:23.400 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:23.399+0000 7f0b0ec12040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T07:13:23.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 Processor -- start 2024-05-09T07:13:23.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 -- start start 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcbc0152a30 0x7fcbc0155f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcbc00c8bc0 0x7fcbc00c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc00c7870 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcbc00c7b50 con 0x7fcbc0152a30 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.436+0000 7fcbc6a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00c7e30 con 0x7fcbc00c5b80 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcbc00c8bc0 0x7fcbc00c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcbc00c8bc0 0x7fcbc00c5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43464/0 (socket says 172.21.15.160:43464) 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/2399960469 learned_addr learned my addr 172.21.15.160:0/2399960469 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbffff640 1 --2- 172.21.15.160:0/2399960469 >> v2:172.21.15.188:3300/0 conn(0x7fcbc0152a30 0x7fcbc0155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbc4ffa640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/2399960469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 msgr2=0x7fcbc0159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/2399960469 >> v2:172.21.15.188:3300/0 conn(0x7fcbc0152a30 msgr2=0x7fcbc0155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 --2- 172.21.15.160:0/2399960469 >> v2:172.21.15.188:3300/0 conn(0x7fcbc0152a30 0x7fcbc0155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/2399960469 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc00c8110 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbc4ffa640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.437+0000 7fcbbf7fe640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc00c5640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcbc008c980 tx=0x7fcbb402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f43b11fad4ac12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbbeffd640 1 -- 172.21.15.160:0/2399960469 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4004030 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbbeffd640 1 -- 172.21.15.160:0/2399960469 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb40041d0 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbbeffd640 1 -- 172.21.15.160:0/2399960469 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4003610 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/2399960469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 msgr2=0x7fcbc00c5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc00c5640 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcbc008c980 tx=0x7fcbb402a1c0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/2399960469 shutdown_connections 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/2399960469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc00c5640 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/2399960469 >> v2:172.21.15.188:3300/0 conn(0x7fcbc0152a30 0x7fcbc0155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/2399960469 >> 172.21.15.160:0/2399960469 conn(0x7fcbc0072ec0 msgr2=0x7fcbc00ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/2399960469 shutdown_connections 2024-05-09T07:13:23.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.438+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/2399960469 wait complete. 2024-05-09T07:13:23.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.439+0000 7fcbc6a5d640 1 Processor -- start 2024-05-09T07:13:23.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 -- start start 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0154b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc0153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbc0152a30 0x7fcbc00abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc00c90d0 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcbc0152e10 con 0x7fcbc0152a30 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbc6a5d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcbc00c9250 con 0x7fcbc00c5b80 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc0153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbbf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc0153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43466/0 (socket says 172.21.15.160:43466) 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.440+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/1885565452 learned_addr learned my addr 172.21.15.160:0/1885565452 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbffff640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0154b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbc4ffa640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbc0152a30 0x7fcbc00abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 msgr2=0x7fcbc0154b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbf7fe640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0154b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbc0152a30 msgr2=0x7fcbc00abaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbf7fe640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbc0152a30 0x7fcbc00abaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbf7fe640 1 -- 172.21.15.160:0/1885565452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbc0154210 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbf7fe640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc0153cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4000c00 tx=0x7fcbb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4041020 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcbb4003c50 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbb4004030 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbc01550f0 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbc01ec800 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.441+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcbb4044020 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.442+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7fcbb403cb20 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.446 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.442+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbc00756c0 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.447 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.444+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcbb404b030 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.506+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbc00c8350 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.506+0000 7fcbbcff9640 1 -- 172.21.15.160:0/1885565452 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7fcbb403cdb0 con 0x7fcbc00c8bc0 2024-05-09T07:13:23.508 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:23.508 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":10,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:23.266097+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 msgr2=0x7fcbc0153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc0153cd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcbb4000c00 tx=0x7fcbb40046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 shutdown_connections 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcbc0152a30 0x7fcbc00abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcbc00c8bc0 0x7fcbc0153cd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 --2- 172.21.15.160:0/1885565452 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcbc00c5b80 0x7fcbc0154b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 >> 172.21.15.160:0/1885565452 conn(0x7fcbc0072ec0 msgr2=0x7fcbc01508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 shutdown_connections 2024-05-09T07:13:23.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:23.507+0000 7fcbc6a5d640 1 -- 172.21.15.160:0/1885565452 wait complete. 2024-05-09T07:13:23.519 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-09T07:13:23.519 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-09T07:13:23.587 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:23.586+0000 7f5cb480d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T07:13:23.640 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:23.638+0000 7f5cb480d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T07:13:23.660 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:23.659+0000 7f0b0ec12040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T07:13:23.714 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:23.712+0000 7f0b0ec12040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T07:13:23.758 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:23.757+0000 7f5cb480d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T07:13:23.827 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:23.826+0000 7f0b0ec12040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T07:13:23.891 INFO:tasks.ceph.osd.0.smithi160.stderr:2024-05-09T07:13:23.890+0000 7f3bb422f640 -1 Falling back to public interface 2024-05-09T07:13:23.898 INFO:tasks.ceph.osd.2.smithi160.stderr:2024-05-09T07:13:23.897+0000 7f2a72eee640 -1 Falling back to public interface 2024-05-09T07:13:23.905 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-09T07:13:23.904+0000 7f7e827fc640 -1 Falling back to public interface 2024-05-09T07:13:23.906 INFO:tasks.ceph.osd.1.smithi160.stderr:2024-05-09T07:13:23.905+0000 7f0cc1944640 -1 Falling back to public interface 2024-05-09T07:13:23.909 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-09T07:13:23.908+0000 7f8e24cdb640 -1 Falling back to public interface 2024-05-09T07:13:23.910 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-05-09T07:13:23.909+0000 7ff79f9fc640 -1 Falling back to public interface 2024-05-09T07:13:23.922 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-09T07:13:23.921+0000 7f425f831640 -1 Falling back to public interface 2024-05-09T07:13:23.931 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-09T07:13:23.929+0000 7f70865a5640 -1 Falling back to public interface 2024-05-09T07:13:24.048 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.046+0000 7f5cb480d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T07:13:24.095 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.094+0000 7f5cb480d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T07:13:24.128 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.127+0000 7f0b0ec12040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T07:13:24.176 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.175+0000 7f0b0ec12040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T07:13:24.252 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.251+0000 7f5cb480d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T07:13:24.326 DEBUG:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:24.332 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.331+0000 7f0b0ec12040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T07:13:24.394 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.390+0000 7f5cb480d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T07:13:24.419 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.418+0000 7f5c9636a640 1 Processor -- start 2024-05-09T07:13:24.419 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.418+0000 7f5c9636a640 1 -- start start 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.418+0000 7f5c9636a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.418+0000 7f5c9636a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5c900c8c00 0x7f5c90158d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c9636a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c900cb1f0 con 0x7f5c90155e60 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c9636a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f5c901563e0 con 0x7f5c900c8c00 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c9636a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5c900cb4d0 con 0x7f5c900c58f0 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5c900c8c00 0x7f5c90158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5c900c8c00 0x7f5c90158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43482/0 (socket says 172.21.15.160:43482) 2024-05-09T07:13:24.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 -- 172.21.15.160:0/3289426351 learned_addr learned my addr 172.21.15.160:0/3289426351 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8ffff640 1 --2- 172.21.15.160:0/3289426351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 -- 172.21.15.160:0/3289426351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 msgr2=0x7f5c901525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 --2- 172.21.15.160:0/3289426351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 -- 172.21.15.160:0/3289426351 >> v1:172.21.15.188:6789/0 conn(0x7f5c90155e60 legacy=0x7f5c901592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.419+0000 7f5c8f7fe640 1 -- 172.21.15.160:0/3289426351 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c900cb7b0 con 0x7f5c900c8c00 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c8f7fe640 1 --2- 172.21.15.160:0/3289426351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c900c8c00 0x7f5c90158d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009870 tx=0x7f5c8402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff99096932118d50 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c8effd640 1 -- 172.21.15.160:0/3289426351 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84037070 con 0x7f5c900c8c00 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c8effd640 1 -- 172.21.15.160:0/3289426351 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c84032440 con 0x7f5c900c8c00 2024-05-09T07:13:24.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c8effd640 1 -- 172.21.15.160:0/3289426351 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84032810 con 0x7f5c900c8c00 2024-05-09T07:13:24.423 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 -- 172.21.15.160:0/3289426351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c900c8c00 msgr2=0x7f5c90158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:24.423 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 --2- 172.21.15.160:0/3289426351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c900c8c00 0x7f5c90158d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5c84009870 tx=0x7f5c8402a1c0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.423 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 -- 172.21.15.160:0/3289426351 shutdown_connections 2024-05-09T07:13:24.423 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 --2- 172.21.15.160:0/3289426351 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c900c8c00 0x7f5c90158d90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.423 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 --2- 172.21.15.160:0/3289426351 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 -- 172.21.15.160:0/3289426351 >> 172.21.15.160:0/3289426351 conn(0x7f5c90061e50 msgr2=0x7f5c90062230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:24.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 -- 172.21.15.160:0/3289426351 shutdown_connections 2024-05-09T07:13:24.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.420+0000 7f5c9636a640 1 -- 172.21.15.160:0/3289426351 wait complete. 2024-05-09T07:13:24.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 Processor -- start 2024-05-09T07:13:24.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 -- start start 2024-05-09T07:13:24.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901e0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:24.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c900c8c00 0x7f5c90154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:24.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 0x7f5c901dee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:24.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900c5e50 con 0x7f5c90155e60 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c900c5cd0 con 0x7f5c900c8c00 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.422+0000 7f5c9636a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5c90152b10 con 0x7f5c900c58f0 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 0x7f5c901dee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8f7fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c900c8c00 0x7f5c90154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 0x7f5c901dee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43484/0 (socket says 172.21.15.160:43484) 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 -- 172.21.15.160:0/1062897475 learned_addr learned my addr 172.21.15.160:0/1062897475 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8ffff640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901e0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 -- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 msgr2=0x7f5c901e0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901e0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 -- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c900c8c00 msgr2=0x7f5c90154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c900c8c00 0x7f5c90154640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 -- 172.21.15.160:0/1062897475 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c901e0c40 con 0x7f5c90155e60 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8f7fe640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c900c8c00 0x7f5c90154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:24.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8ffff640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c94907640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 0x7f5c901dee70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c6b0 tx=0x7f5c8800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88003b70 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c88003d30 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c901e0ee0 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c88005310 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.423+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c901568e0 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.424+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5c880054b0 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.424+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5c88012b90 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.425+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c90075530 con 0x7f5c90155e60 2024-05-09T07:13:24.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.427+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f5c880124c0 con 0x7f5c90155e60 2024-05-09T07:13:24.457 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.456+0000 7f5cb480d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T07:13:24.469 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.467+0000 7f0b0ec12040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T07:13:24.504 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.503+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c9006d4d0 con 0x7f5c90155e60 2024-05-09T07:13:24.505 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.503+0000 7f5c8cff9640 1 -- 172.21.15.160:0/1062897475 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5c880126a0 con 0x7f5c90155e60 2024-05-09T07:13:24.505 INFO:teuthology.misc.health.smithi160.stdout: 2024-05-09T07:13:24.505 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":10,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:23.266097+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 msgr2=0x7f5c901dee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 0x7f5c901dee70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5c8800c6b0 tx=0x7f5c8800cb70 comp rx=0 tx=0).stop 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 shutdown_connections 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c900c8c00 0x7f5c90154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c90155e60 0x7f5c901dee70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 --2- 172.21.15.160:0/1062897475 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c900c58f0 0x7f5c901e0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 >> 172.21.15.160:0/1062897475 conn(0x7f5c90061e50 msgr2=0x7f5c900c9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:24.506 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 shutdown_connections 2024-05-09T07:13:24.507 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:24.504+0000 7f5c9636a640 1 -- 172.21.15.160:0/1062897475 wait complete. 2024-05-09T07:13:24.515 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T07:13:24.519 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.518+0000 7f0b0ec12040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T07:13:24.530 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.529+0000 7f5cb480d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T07:13:24.576 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:24.575+0000 7f5cb480d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T07:13:24.587 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.586+0000 7f0b0ec12040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T07:13:24.632 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:24.630+0000 7f0b0ec12040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T07:13:25.052 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:25.051+0000 7f5cb480d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T07:13:25.105 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:25.104+0000 7f0b0ec12040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T07:13:25.167 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:25.166+0000 7f5cb480d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T07:13:25.226 INFO:tasks.ceph.mgr.x.smithi188.stderr:2024-05-09T07:13:25.224+0000 7f0b0ec12040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T07:13:26.187 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:26.186+0000 7f5c54fd9640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T07:13:26.187 INFO:tasks.ceph.mgr.y.smithi160.stderr:2024-05-09T07:13:26.186+0000 7f5c54fd9640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-09T07:13:27.390 INFO:tasks.ceph.osd.0.smithi160.stderr:2024-05-09T07:13:27.389+0000 7f3bb422f640 -1 osd.0 0 log_to_monitors true 2024-05-09T07:13:27.391 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-09T07:13:27.389+0000 7f8e24cdb640 -1 osd.7 0 log_to_monitors true 2024-05-09T07:13:27.394 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-09T07:13:27.393+0000 7f7e827fc640 -1 osd.4 0 log_to_monitors true 2024-05-09T07:13:27.425 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-09T07:13:27.424+0000 7f425f831640 -1 osd.5 0 log_to_monitors true 2024-05-09T07:13:27.429 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-09T07:13:27.428+0000 7f70865a5640 -1 osd.6 0 log_to_monitors true 2024-05-09T07:13:27.431 INFO:tasks.ceph.osd.2.smithi160.stderr:2024-05-09T07:13:27.430+0000 7f2a72eee640 -1 osd.2 0 log_to_monitors true 2024-05-09T07:13:27.440 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-05-09T07:13:27.438+0000 7ff79f9fc640 -1 osd.3 0 log_to_monitors true 2024-05-09T07:13:27.445 INFO:tasks.ceph.osd.1.smithi160.stderr:2024-05-09T07:13:27.443+0000 7f0cc1944640 -1 osd.1 0 log_to_monitors true 2024-05-09T07:13:31.278 INFO:tasks.ceph.osd.1.smithi160.stderr:2024-05-09T07:13:31.276+0000 7f0cbd8da640 -1 osd.1 0 waiting for initial osdmap 2024-05-09T07:13:31.280 INFO:tasks.ceph.osd.0.smithi160.stderr:2024-05-09T07:13:31.278+0000 7f3baff94640 -1 osd.0 0 waiting for initial osdmap 2024-05-09T07:13:31.294 INFO:tasks.ceph.osd.1.smithi160.stderr:2024-05-09T07:13:31.293+0000 7f0cb56fa640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.295 INFO:tasks.ceph.osd.0.smithi160.stderr:2024-05-09T07:13:31.293+0000 7f3ba75b1640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.296 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-09T07:13:31.295+0000 7f7e7efa6640 -1 osd.4 0 waiting for initial osdmap 2024-05-09T07:13:31.309 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-09T07:13:31.307+0000 7f425ad80640 -1 osd.5 0 waiting for initial osdmap 2024-05-09T07:13:31.311 INFO:tasks.ceph.osd.4.smithi188.stderr:2024-05-09T07:13:31.310+0000 7f7e765b2640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.313 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-09T07:13:31.311+0000 7f8e20c71640 -1 osd.7 0 waiting for initial osdmap 2024-05-09T07:13:31.321 DEBUG:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:31.322 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-09T07:13:31.320+0000 7f708253b640 -1 osd.6 0 waiting for initial osdmap 2024-05-09T07:13:31.323 INFO:tasks.ceph.osd.7.smithi188.stderr:2024-05-09T07:13:31.322+0000 7f8e18a91640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.325 INFO:tasks.ceph.osd.5.smithi188.stderr:2024-05-09T07:13:31.323+0000 7f4252bb1640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.336 INFO:tasks.ceph.osd.6.smithi188.stderr:2024-05-09T07:13:31.335+0000 7f707a35b640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.350 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-05-09T07:13:31.348+0000 7ff79b992640 -1 osd.3 0 waiting for initial osdmap 2024-05-09T07:13:31.368 INFO:tasks.ceph.osd.2.smithi160.stderr:2024-05-09T07:13:31.365+0000 7f2a6ee84640 -1 osd.2 0 waiting for initial osdmap 2024-05-09T07:13:31.369 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-05-09T07:13:31.368+0000 7ff7937b2640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.389 INFO:tasks.ceph.osd.2.smithi160.stderr:2024-05-09T07:13:31.388+0000 7f2a66ca4640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T07:13:31.419 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.417+0000 7f947b36a640 1 Processor -- start 2024-05-09T07:13:31.419 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.417+0000 7f947b36a640 1 -- start start 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f947b36a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f94741541f0 0x7f947406b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f947b36a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f9474157630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f947b36a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94740c7020 con 0x7f947406b790 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f947b36a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f94740c64a0 con 0x7f94741541f0 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f947b36a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f94740c2f60 con 0x7f947406be80 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f9479106640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f94741541f0 0x7f947406b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f9479106640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f94741541f0 0x7f947406b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43582/0 (socket says 172.21.15.160:43582) 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f9479106640 1 -- 172.21.15.160:0/867323424 learned_addr learned my addr 172.21.15.160:0/867323424 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.418+0000 7f9479907640 1 --2- 172.21.15.160:0/867323424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f9474157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f9479106640 1 -- 172.21.15.160:0/867323424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 msgr2=0x7f9474157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f9479106640 1 --2- 172.21.15.160:0/867323424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f9474157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f9479106640 1 -- 172.21.15.160:0/867323424 >> v1:172.21.15.188:6789/0 conn(0x7f947406b790 legacy=0x7f9474156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:31.421 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f9479106640 1 -- 172.21.15.160:0/867323424 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94740c66f0 con 0x7f94741541f0 2024-05-09T07:13:31.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f9479106640 1 --2- 172.21.15.160:0/867323424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94741541f0 0x7f947406b250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9470009870 tx=0x7f947002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a34e0c76250d5c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:31.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f946bfff640 1 -- 172.21.15.160:0/867323424 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470037070 con 0x7f94741541f0 2024-05-09T07:13:31.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f946bfff640 1 -- 172.21.15.160:0/867323424 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9470032440 con 0x7f94741541f0 2024-05-09T07:13:31.422 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.419+0000 7f946bfff640 1 -- 172.21.15.160:0/867323424 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470032810 con 0x7f94741541f0 2024-05-09T07:13:31.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 -- 172.21.15.160:0/867323424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94741541f0 msgr2=0x7f947406b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:31.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 --2- 172.21.15.160:0/867323424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94741541f0 0x7f947406b250 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9470009870 tx=0x7f947002a270 comp rx=0 tx=0).stop 2024-05-09T07:13:31.424 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 -- 172.21.15.160:0/867323424 shutdown_connections 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 --2- 172.21.15.160:0/867323424 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f9474157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 --2- 172.21.15.160:0/867323424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f94741541f0 0x7f947406b250 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 -- 172.21.15.160:0/867323424 >> 172.21.15.160:0/867323424 conn(0x7f94740a9820 msgr2=0x7f94740a9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 -- 172.21.15.160:0/867323424 shutdown_connections 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.420+0000 7f947b36a640 1 -- 172.21.15.160:0/867323424 wait complete. 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.421+0000 7f947b36a640 1 Processor -- start 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.421+0000 7f947b36a640 1 -- start start 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f947b36a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 0x7f947414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f947b36a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f94740c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f947b36a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94741541f0 0x7f947414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f947b36a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f94741545d0 con 0x7f947406b790 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f947b36a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f94740c6cb0 con 0x7f94741541f0 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f947b36a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9474073510 con 0x7f947406be80 2024-05-09T07:13:31.425 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 0x7f947414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 0x7f947414e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43588/0 (socket says 172.21.15.160:43588) 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 -- 172.21.15.160:0/3088879313 learned_addr learned my addr 172.21.15.160:0/3088879313 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 -- 172.21.15.160:0/3088879313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 msgr2=0x7f94740c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f94740c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 -- 172.21.15.160:0/3088879313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94741541f0 msgr2=0x7f947414aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94741541f0 0x7f947414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 -- 172.21.15.160:0/3088879313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947414e7a0 con 0x7f947406b790 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9479106640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 0x7f947414e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9470000c00 tx=0x7f94700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.422+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470037040 con 0x7f947406b790 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.423+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9470042070 con 0x7f947406b790 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.423+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9474155200 con 0x7f947406b790 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.423+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947003b870 con 0x7f947406b790 2024-05-09T07:13:31.426 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.423+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9474155f10 con 0x7f947406b790 2024-05-09T07:13:31.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.424+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9474155d30 con 0x7f947406b790 2024-05-09T07:13:31.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.424+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 4) ==== 90442+0+0 (secure 0 0 0) 0x7f947003ba10 con 0x7f947406b790 2024-05-09T07:13:31.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.425+0000 7f9469ffb640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f945806bda0 0x7f945806e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:31.427 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.425+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2231+0+0 (secure 0 0 0) 0x7f94700aa710 con 0x7f947406b790 2024-05-09T07:13:31.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.425+0000 7f9478905640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f945806bda0 0x7f945806e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:31.428 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.425+0000 7f9478905640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f945806bda0 0x7f945806e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9464000f60 tx=0x7f94640079d0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:31.430 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.429+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9474155d30 con 0x7f947406b790 2024-05-09T07:13:31.604 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.603+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9474075530 con 0x7f947406b790 2024-05-09T07:13:31.605 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.603+0000 7f9469ffb640 1 -- 172.21.15.160:0/3088879313 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) ==== 74+0+7058 (secure 0 0 0) 0x7f9470079fc0 con 0x7f947406b790 2024-05-09T07:13:31.605 INFO:teuthology.misc.health.smithi160.stdout: 2024-05-09T07:13:31.605 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":12,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:29.190084+0000","last_up_change":"0.000000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f945806bda0 msgr2=0x7f945806e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f945806bda0 0x7f945806e260 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9464000f60 tx=0x7f94640079d0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 msgr2=0x7f947414e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 0x7f947414e260 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9470000c00 tx=0x7f94700046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 shutdown_connections 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f945806bda0 0x7f945806e260 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.607 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f94741541f0 0x7f947414aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.608 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f947406be80 0x7f94740c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.608 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 --2- 172.21.15.160:0/3088879313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f947406b790 0x7f947414e260 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:31.608 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.605+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 >> 172.21.15.160:0/3088879313 conn(0x7f94740a9820 msgr2=0x7f9474156f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:31.608 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.606+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 shutdown_connections 2024-05-09T07:13:31.608 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:31.606+0000 7f947b36a640 1 -- 172.21.15.160:0/3088879313 wait complete. 2024-05-09T07:13:31.616 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-09T07:13:38.424 DEBUG:teuthology.orchestra.run.smithi160:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:38.527 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.525+0000 7f8d39b1d640 1 Processor -- start 2024-05-09T07:13:38.527 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.525+0000 7f8d39b1d640 1 -- start start 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d39b1d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8d340c4b80 0x7f8d3406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d39b1d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d34152c90 con 0x7f8d3406e070 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d39b1d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8d340c1460 con 0x7f8d340a62b0 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d39b1d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8d3406ced0 con 0x7f8d340c4b80 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d33fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8d340c4b80 0x7f8d3406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d337fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8d3406e070 0x7f8d3406d360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51234/0 (socket says 172.21.15.160:51234) 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.526+0000 7f8d337fe640 1 -- 172.21.15.160:0/3916273247 learned_addr learned my addr 172.21.15.160:0/3916273247 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d33fff640 1 -- 172.21.15.160:0/3916273247 >> v1:172.21.15.188:6789/0 conn(0x7f8d340a62b0 legacy=0x7f8d3415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d33fff640 1 -- 172.21.15.160:0/3916273247 >> v1:172.21.15.160:6789/0 conn(0x7f8d3406e070 legacy=0x7f8d3406d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d327fc640 1 -- 172.21.15.160:0/3916273247 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 375908420 0 0) 0x7f8d34152c90 con 0x7f8d3406e070 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d33fff640 1 -- 172.21.15.160:0/3916273247 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d34152f30 con 0x7f8d340c4b80 2024-05-09T07:13:38.529 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d33fff640 1 --2- 172.21.15.160:0/3916273247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d340c4b80 0x7f8d3406da70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0097a0 tx=0x7f8d2c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ef96463d276f4cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:38.530 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d327fc640 1 -- 172.21.15.160:0/3916273247 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c037070 con 0x7f8d340c4b80 2024-05-09T07:13:38.530 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d327fc640 1 -- 172.21.15.160:0/3916273247 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d2c032440 con 0x7f8d340c4b80 2024-05-09T07:13:38.530 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.527+0000 7f8d327fc640 1 -- 172.21.15.160:0/3916273247 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d2c02ae60 con 0x7f8d340c4b80 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 -- 172.21.15.160:0/3916273247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d340c4b80 msgr2=0x7f8d3406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/3916273247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d340c4b80 0x7f8d3406da70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c0097a0 tx=0x7f8d2c02a110 comp rx=0 tx=0).stop 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 -- 172.21.15.160:0/3916273247 shutdown_connections 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/3916273247 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d340c4b80 0x7f8d3406da70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 -- 172.21.15.160:0/3916273247 >> 172.21.15.160:0/3916273247 conn(0x7f8d340a8810 msgr2=0x7f8d340a8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 -- 172.21.15.160:0/3916273247 shutdown_connections 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.528+0000 7f8d39b1d640 1 -- 172.21.15.160:0/3916273247 wait complete. 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.529+0000 7f8d39b1d640 1 Processor -- start 2024-05-09T07:13:38.531 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.529+0000 7f8d39b1d640 1 -- start start 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d39b1d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 0x7f8d3414ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d39b1d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d340a62b0 0x7f8d341debb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d39b1d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d340c4b80 0x7f8d34148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d39b1d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3414faa0 con 0x7f8d3406e070 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d39b1d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8d340a5cd0 con 0x7f8d340a62b0 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d39b1d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8d341ea5d0 con 0x7f8d340c4b80 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d337fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 0x7f8d3414ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d337fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 0x7f8d3414ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43298/0 (socket says 172.21.15.160:43298) 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.530+0000 7f8d337fe640 1 -- 172.21.15.160:0/722430635 learned_addr learned my addr 172.21.15.160:0/722430635 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d337fe640 1 -- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d340c4b80 msgr2=0x7f8d34148cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d32ffd640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d340a62b0 0x7f8d341debb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.534 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d33fff640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d340c4b80 0x7f8d34148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d337fe640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d340c4b80 0x7f8d34148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d337fe640 1 -- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d340a62b0 msgr2=0x7f8d341debb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d337fe640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d340a62b0 0x7f8d341debb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d337fe640 1 -- 172.21.15.160:0/722430635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3414e090 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d337fe640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 0x7f8d3414ecb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c3f0 tx=0x7f8d2400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d240074e0 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8d240047e0 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d341ea820 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24004bd0 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.531+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d341eade0 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.532+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d34075530 con 0x7f8d3406e070 2024-05-09T07:13:38.535 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.532+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8d24011070 con 0x7f8d3406e070 2024-05-09T07:13:38.536 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.533+0000 7f8d2bfff640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8d1406be30 0x7f8d1406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.536 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.533+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f8d2408a6b0 con 0x7f8d3406e070 2024-05-09T07:13:38.536 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.533+0000 7f8d32ffd640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8d1406be30 0x7f8d1406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.536 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.533+0000 7f8d32ffd640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8d1406be30 0x7f8d1406e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007fd0 tx=0x7f8d1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:38.539 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.538+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8d24059440 con 0x7f8d3406e070 2024-05-09T07:13:38.711 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.710+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8d34070a60 con 0x7f8d3406e070 2024-05-09T07:13:38.712 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.711+0000 7f8d2bfff640 1 -- 172.21.15.160:0/722430635 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f8d24058d40 con 0x7f8d3406e070 2024-05-09T07:13:38.712 INFO:teuthology.misc.health.smithi160.stdout: 2024-05-09T07:13:38.712 INFO:teuthology.misc.health.smithi160.stdout:{"epoch":16,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:35.358666+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.712+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8d1406be30 msgr2=0x7f8d1406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.712+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8d1406be30 0x7f8d1406e2f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c007fd0 tx=0x7f8d1c009040 comp rx=0 tx=0).stop 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.712+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 msgr2=0x7f8d3414ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.712+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 0x7f8d3414ecb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8d2400c3f0 tx=0x7f8d2400c8b0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 shutdown_connections 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8d1406be30 0x7f8d1406e2f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8d340a62b0 0x7f8d341debb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.718 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8d340c4b80 0x7f8d34148cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.719 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 --2- 172.21.15.160:0/722430635 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8d3406e070 0x7f8d3414ecb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.719 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 >> 172.21.15.160:0/722430635 conn(0x7f8d340a8810 msgr2=0x7f8d340c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:38.719 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 shutdown_connections 2024-05-09T07:13:38.719 INFO:teuthology.misc.health.smithi160.stderr:2024-05-09T07:13:38.713+0000 7f8d39b1d640 1 -- 172.21.15.160:0/722430635 wait complete. 2024-05-09T07:13:38.724 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-09T07:13:38.724 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-09T07:13:38.725 INFO:tasks.ceph.mds.a:Restarting daemon 2024-05-09T07:13:38.725 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-05-09T07:13:38.727 INFO:tasks.ceph.mds.a:Started 2024-05-09T07:13:38.727 INFO:tasks.ceph.mds.c:Restarting daemon 2024-05-09T07:13:38.728 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-05-09T07:13:38.729 INFO:tasks.ceph.mds.c:Started 2024-05-09T07:13:38.730 INFO:tasks.ceph.mds.b:Restarting daemon 2024-05-09T07:13:38.730 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-05-09T07:13:38.732 INFO:tasks.ceph.mds.b:Started 2024-05-09T07:13:38.733 INFO:tasks.ceph.mds.d:Restarting daemon 2024-05-09T07:13:38.733 DEBUG:teuthology.orchestra.run.smithi188:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-05-09T07:13:38.735 INFO:tasks.ceph.mds.d:Started 2024-05-09T07:13:38.735 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-05-09T07:13:38.735 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:38.803 INFO:tasks.ceph.mds.c.smithi160.stdout:starting mds.c at 2024-05-09T07:13:38.803 INFO:tasks.ceph.mds.a.smithi160.stdout:starting mds.a at 2024-05-09T07:13:38.817 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-05-09T07:13:38.822 INFO:tasks.ceph.mds.b.smithi188.stdout:starting mds.b at 2024-05-09T07:13:38.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.840+0000 7f5e51872640 1 Processor -- start 2024-05-09T07:13:38.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.840+0000 7f5e51872640 1 -- start start 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e51872640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 0x7f5e4c0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e51872640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0c8b90 0x7f5e4c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e51872640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e4c159840 con 0x7f5e4c0c5610 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e51872640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5e4c159310 con 0x7f5e4c0c8b90 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e51872640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c159560 con 0x7f5e4c0be320 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4affd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5e4c0c5610 0x7f5e4c155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51274/0 (socket says 172.21.15.160:51274) 2024-05-09T07:13:38.845 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4affd640 1 -- 172.21.15.160:0/2035441872 learned_addr learned my addr 172.21.15.160:0/2035441872 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4b7fe640 1 --2- 172.21.15.160:0/2035441872 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0c8b90 0x7f5e4c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e43fff640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 0x7f5e4c0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3614078773 0 0) 0x7f5e4c159840 con 0x7f5e4c0c5610 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e38003520 con 0x7f5e4c0c5610 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3946280967 0 0) 0x7f5e38003520 con 0x7f5e4c0c5610 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 msgr2=0x7f5e4c0c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 0x7f5e4c0c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0c8b90 msgr2=0x7f5e4c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 --2- 172.21.15.160:0/2035441872 >> v2:172.21.15.188:3300/0 conn(0x7f5e4c0c8b90 0x7f5e4c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c0c77d0 con 0x7f5e4c0c5610 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f5e3c001670 con 0x7f5e4c0c5610 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 >> v1:172.21.15.160:6789/0 conn(0x7f5e4c0c5610 legacy=0x7f5e4c155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.841+0000 7f5e4a7fc640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380039b0 0x7f5e38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4a7fc640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e38004800 0x7f5e38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4a7fc640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 0x7f5e380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c159310 con 0x7f5e38004800 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c159560 con 0x7f5e380039b0 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e3800baa0 con 0x7f5e4c0be320 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e38004800 0x7f5e38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4affd640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380039b0 0x7f5e38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e4b7fe640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 0x7f5e380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 -- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 msgr2=0x7f5e380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e4c0be320 0x7f5e380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 -- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380039b0 msgr2=0x7f5e38003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e380039b0 0x7f5e38003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 -- 172.21.15.160:0/2035441872 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e38003520 con 0x7f5e38004800 2024-05-09T07:13:38.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.842+0000 7f5e43fff640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e38004800 0x7f5e38004c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0be760 tx=0x7f5e38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac9c37d7d5f2bb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34036070 con 0x7f5e38004800 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e34031400 con 0x7f5e38004800 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e4a7fc640 1 -- 172.21.15.160:0/2035441872 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e34009690 con 0x7f5e38004800 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e51872640 1 -- 172.21.15.160:0/2035441872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e38004800 msgr2=0x7f5e38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e51872640 1 --2- 172.21.15.160:0/2035441872 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e38004800 0x7f5e38004c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c0be760 tx=0x7f5e38005830 comp rx=0 tx=0).stop 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e4b7fe640 1 -- 172.21.15.160:0/2035441872 reap_dead start 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e51872640 1 -- 172.21.15.160:0/2035441872 shutdown_connections 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e51872640 1 -- 172.21.15.160:0/2035441872 >> 172.21.15.160:0/2035441872 conn(0x7f5e4c072e90 msgr2=0x7f5e4c0ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e51872640 1 -- 172.21.15.160:0/2035441872 shutdown_connections 2024-05-09T07:13:38.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.843+0000 7f5e51872640 1 -- 172.21.15.160:0/2035441872 wait complete. 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.844+0000 7f5e51872640 1 Processor -- start 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.844+0000 7f5e51872640 1 -- start start 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e51872640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e38004800 0x7f5e4c0be700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e51872640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c1eb670 0x7f5e4c154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e51872640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 0x7f5e4c145300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e51872640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0ad6a0 con 0x7f5e4c156a00 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e51872640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5e4c0c7d90 con 0x7f5e4c1eb670 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e51872640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5e4c155f60 con 0x7f5e38004800 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 0x7f5e4c145300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 0x7f5e4c145300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43356/0 (socket says 172.21.15.160:43356) 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 -- 172.21.15.160:0/456521376 learned_addr learned my addr 172.21.15.160:0/456521376 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:38.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e43fff640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c1eb670 0x7f5e4c154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4affd640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e38004800 0x7f5e4c0be700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 -- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e38004800 msgr2=0x7f5e4c0be700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e38004800 0x7f5e4c0be700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 -- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c1eb670 msgr2=0x7f5e4c154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c1eb670 0x7f5e4c154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.845+0000 7f5e4b7fe640 1 -- 172.21.15.160:0/456521376 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e4c156270 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.846+0000 7f5e4b7fe640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 0x7f5e4c145300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e44007d10 tx=0x7f5e440115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.846+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44011de0 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.846+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e4401b070 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.846+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e4c155040 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.846+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e44003990 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.846+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e4c157e60 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.847+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e44005d10 con 0x7f5e4c156a00 2024-05-09T07:13:38.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.847+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e4c0756c0 con 0x7f5e4c156a00 2024-05-09T07:13:38.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.848+0000 7f5e48ff9640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e2c06be30 0x7f5e2c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:38.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.848+0000 7f5e4affd640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e2c06be30 0x7f5e2c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:38.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.848+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f5e4408b0e0 con 0x7f5e4c156a00 2024-05-09T07:13:38.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.848+0000 7f5e4affd640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e2c06be30 0x7f5e2c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c007ac0 tx=0x7f5e3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:38.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:38.852+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e44025020 con 0x7f5e4c156a00 2024-05-09T07:13:39.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.026+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e4c0ad340 con 0x7f5e4c156a00 2024-05-09T07:13:39.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.026+0000 7f5e48ff9640 1 -- 172.21.15.160:0/456521376 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f5e44059de0 con 0x7f5e4c156a00 2024-05-09T07:13:39.028 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:39.028 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":16,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:35.358666+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:39.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.028+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e2c06be30 msgr2=0x7f5e2c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.028+0000 7f5e51872640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e2c06be30 0x7f5e2c06e2f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c007ac0 tx=0x7f5e3c009040 comp rx=0 tx=0).stop 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.028+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 msgr2=0x7f5e4c145300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.028+0000 7f5e51872640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 0x7f5e4c145300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f5e44007d10 tx=0x7f5e440115a0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 shutdown_connections 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e2c06be30 0x7f5e2c06e2f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5e4c1eb670 0x7f5e4c154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e4c156a00 0x7f5e4c145300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 --2- 172.21.15.160:0/456521376 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5e38004800 0x7f5e4c0be700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 >> 172.21.15.160:0/456521376 conn(0x7f5e4c072e90 msgr2=0x7f5e4c151b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 shutdown_connections 2024-05-09T07:13:39.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.029+0000 7f5e51872640 1 -- 172.21.15.160:0/456521376 wait complete. 2024-05-09T07:13:39.041 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T07:13:39.041 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:13:39.144 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.143+0000 7f89a9da5640 1 Processor -- start 2024-05-09T07:13:39.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.143+0000 7f89a9da5640 1 -- start start 2024-05-09T07:13:39.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.143+0000 7f89a9da5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a41568e0 0x7f89a4159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.143+0000 7f89a9da5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a4156d90 con 0x7f89a4152a80 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.143+0000 7f89a9da5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f89a40c7ae0 con 0x7f89a40c5690 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.143+0000 7f89a9da5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f89a4152e60 con 0x7f89a41568e0 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a37fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f89a4152a80 0x7f89a4155e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51284/0 (socket says 172.21.15.160:51284) 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a37fe640 1 -- 172.21.15.160:0/2196979753 learned_addr learned my addr 172.21.15.160:0/2196979753 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1122384413 0 0) 0x7f89a4156d90 con 0x7f89a4152a80 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f898c003520 con 0x7f89a4152a80 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a41568e0 0x7f89a4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3066323580 0 0) 0x7f898c003520 con 0x7f89a4152a80 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a41568e0 msgr2=0x7f89a4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a41568e0 0x7f89a4159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 >> v1:172.21.15.188:6789/0 conn(0x7f89a40c5690 legacy=0x7f89a415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a40c7dc0 con 0x7f89a4152a80 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f8998001670 con 0x7f89a4152a80 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 >> v1:172.21.15.160:6789/0 conn(0x7f89a4152a80 legacy=0x7f89a4155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f898c0039e0 0x7f898c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f898c004800 0x7f898c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f898c005290 0x7f898c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4152e60 con 0x7f898c005290 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f898c00c1b0 con 0x7f898c004800 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f898c00c3b0 con 0x7f898c0039e0 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a41568e0 0x7f89a4159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a3fff640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f898c005290 0x7f898c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a37fe640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f898c0039e0 0x7f898c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f898c004800 0x7f898c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.150 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a3fff640 1 -- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f898c0039e0 msgr2=0x7f898c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a3fff640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f898c0039e0 0x7f898c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a3fff640 1 -- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f898c004800 msgr2=0x7f898c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a3fff640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f898c004800 0x7f898c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a3fff640 1 -- 172.21.15.160:0/2196979753 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c003520 con 0x7f898c005290 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.144+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f898c004800 0x7f898c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.145+0000 7f89a3fff640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f898c005290 0x7f898c00b9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f899c001e50 tx=0x7f898c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d1d210bb257d2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.145+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c001a70 con 0x7f898c005290 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.145+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899c003970 con 0x7f898c005290 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.145+0000 7f89a27fc640 1 -- 172.21.15.160:0/2196979753 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899c003d60 con 0x7f898c005290 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a9da5640 1 -- 172.21.15.160:0/2196979753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f898c005290 msgr2=0x7f898c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a9da5640 1 --2- 172.21.15.160:0/2196979753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f898c005290 0x7f898c00b9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f899c001e50 tx=0x7f898c005830 comp rx=0 tx=0).stop 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a3fff640 1 -- 172.21.15.160:0/2196979753 reap_dead start 2024-05-09T07:13:39.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a9da5640 1 -- 172.21.15.160:0/2196979753 shutdown_connections 2024-05-09T07:13:39.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a9da5640 1 -- 172.21.15.160:0/2196979753 >> 172.21.15.160:0/2196979753 conn(0x7f89a4072ec0 msgr2=0x7f89a40ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:39.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a9da5640 1 -- 172.21.15.160:0/2196979753 shutdown_connections 2024-05-09T07:13:39.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.146+0000 7f89a9da5640 1 -- 172.21.15.160:0/2196979753 wait complete. 2024-05-09T07:13:39.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.147+0000 7f89a9da5640 1 Processor -- start 2024-05-09T07:13:39.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.147+0000 7f89a9da5640 1 -- start start 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a414ece0 0x7f89a414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 0x7f89a414e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a41497e0 0x7f89a41485e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f89a4158df0 con 0x7f89a40c8af0 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f89a40c80a0 con 0x7f89a41497e0 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f89a40ad6a0 con 0x7f89a414ece0 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 0x7f89a414e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 0x7f89a414e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43376/0 (socket says 172.21.15.160:43376) 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 -- 172.21.15.160:0/1310649801 learned_addr learned my addr 172.21.15.160:0/1310649801 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 -- 172.21.15.160:0/1310649801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a414ece0 msgr2=0x7f89a414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a414ece0 0x7f89a414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 -- 172.21.15.160:0/1310649801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a41497e0 msgr2=0x7f89a41485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a41497e0 0x7f89a41485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 -- 172.21.15.160:0/1310649801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89a41495a0 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a2ffd640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 0x7f89a414e4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8994010840 tx=0x7f8994010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899401a020 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f899400e870 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89a414cec0 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f899400f040 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.148+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89a414d480 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.149+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89a40c85c0 con 0x7f89a40c8af0 2024-05-09T07:13:39.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.150+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f899401d020 con 0x7f89a40c8af0 2024-05-09T07:13:39.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.150+0000 7f898bfff640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f899006be60 0x7f899006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.150+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f899408b240 con 0x7f89a40c8af0 2024-05-09T07:13:39.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.150+0000 7f89a37fe640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f899006be60 0x7f899006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.151+0000 7f89a37fe640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f899006be60 0x7f899006e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8998001b80 tx=0x7f8998001b10 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:39.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.155+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8994024020 con 0x7f89a40c8af0 2024-05-09T07:13:39.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.332+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f89a40ad3a0 con 0x7f89a40c8af0 2024-05-09T07:13:39.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.332+0000 7f898bfff640 1 -- 172.21.15.160:0/1310649801 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f8994059fd0 con 0x7f89a40c8af0 2024-05-09T07:13:39.334 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f899006be60 msgr2=0x7f899006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f899006be60 0x7f899006e320 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8998001b80 tx=0x7f8998001b10 comp rx=0 tx=0).stop 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 msgr2=0x7f89a414e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 0x7f89a414e4a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8994010840 tx=0x7f8994010d00 comp rx=0 tx=0).stop 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 shutdown_connections 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f899006be60 0x7f899006e320 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f89a41497e0 0x7f89a41485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f89a40c8af0 0x7f89a414e4a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 --2- 172.21.15.160:0/1310649801 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f89a414ece0 0x7f89a414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 >> 172.21.15.160:0/1310649801 conn(0x7f89a4072ec0 msgr2=0x7f89a406c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:39.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 shutdown_connections 2024-05-09T07:13:39.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.334+0000 7f89a9da5640 1 -- 172.21.15.160:0/1310649801 wait complete. 2024-05-09T07:13:39.347 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:13:39.447 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 Processor -- start 2024-05-09T07:13:39.447 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 -- start start 2024-05-09T07:13:39.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fbd381561f0 0x7fbd38159640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbd380c5f20 0x7fbd3806b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd380c6450 con 0x7fbd3806b6a0 2024-05-09T07:13:39.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fbd380c2e60 con 0x7fbd380c5f20 2024-05-09T07:13:39.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3e75a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fbd380c6730 con 0x7fbd381561f0 2024-05-09T07:13:39.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd2ffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbd380c5f20 0x7fbd3806b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3ccf7640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fbd3806b6a0 0x7fbd380c32f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51296/0 (socket says 172.21.15.160:51296) 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.446+0000 7fbd3ccf7640 1 -- 172.21.15.160:0/4274931097 learned_addr learned my addr 172.21.15.160:0/4274931097 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd2ffff640 1 -- 172.21.15.160:0/4274931097 >> v2:172.21.15.188:3300/0 conn(0x7fbd381561f0 msgr2=0x7fbd38159640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd37fff640 1 --2- 172.21.15.160:0/4274931097 >> v2:172.21.15.188:3300/0 conn(0x7fbd381561f0 0x7fbd38159640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd377fe640 1 -- 172.21.15.160:0/4274931097 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 875211472 0 0) 0x7fbd380c6450 con 0x7fbd3806b6a0 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd2ffff640 1 --2- 172.21.15.160:0/4274931097 >> v2:172.21.15.188:3300/0 conn(0x7fbd381561f0 0x7fbd38159640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd2ffff640 1 -- 172.21.15.160:0/4274931097 >> v1:172.21.15.160:6789/0 conn(0x7fbd3806b6a0 legacy=0x7fbd380c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd2ffff640 1 -- 172.21.15.160:0/4274931097 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd380c6a10 con 0x7fbd380c5f20 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.447+0000 7fbd2ffff640 1 --2- 172.21.15.160:0/4274931097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd380c5f20 0x7fbd3806b160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd380c5610 tx=0x7fbd2002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a72765e57a8a9d7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd377fe640 1 -- 172.21.15.160:0/4274931097 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20037070 con 0x7fbd380c5f20 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd377fe640 1 -- 172.21.15.160:0/4274931097 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd20032440 con 0x7fbd380c5f20 2024-05-09T07:13:39.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd377fe640 1 -- 172.21.15.160:0/4274931097 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20032810 con 0x7fbd380c5f20 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 -- 172.21.15.160:0/4274931097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd380c5f20 msgr2=0x7fbd3806b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/4274931097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd380c5f20 0x7fbd3806b160 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbd380c5610 tx=0x7fbd2002a1a0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 -- 172.21.15.160:0/4274931097 shutdown_connections 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/4274931097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd380c5f20 0x7fbd3806b160 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/4274931097 >> v2:172.21.15.188:3300/0 conn(0x7fbd381561f0 0x7fbd38159640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 -- 172.21.15.160:0/4274931097 >> 172.21.15.160:0/4274931097 conn(0x7fbd380a9800 msgr2=0x7fbd380a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:39.453 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 -- 172.21.15.160:0/4274931097 shutdown_connections 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.448+0000 7fbd3e75a640 1 -- 172.21.15.160:0/4274931097 wait complete. 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.449+0000 7fbd3e75a640 1 Processor -- start 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.449+0000 7fbd3e75a640 1 -- start start 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3e75a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3806b6a0 0x7fbd38159310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3e75a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd380c5f20 0x7fbd3806e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3e75a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 0x7fbd3814ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3e75a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3806fa90 con 0x7fbd381561f0 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3e75a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbd381565d0 con 0x7fbd3806b6a0 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3e75a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3806fc10 con 0x7fbd380c5f20 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 0x7fbd3814ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 0x7fbd3814ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43394/0 (socket says 172.21.15.160:43394) 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 -- 172.21.15.160:0/1906601604 learned_addr learned my addr 172.21.15.160:0/1906601604 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:39.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd2ffff640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd380c5f20 0x7fbd3806e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd37fff640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3806b6a0 0x7fbd38159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 -- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd380c5f20 msgr2=0x7fbd3806e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd380c5f20 0x7fbd3806e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 -- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3806b6a0 msgr2=0x7fbd38159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3806b6a0 0x7fbd38159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.450+0000 7fbd3ccf7640 1 -- 172.21.15.160:0/1906601604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd380accf0 con 0x7fbd381561f0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.451+0000 7fbd3ccf7640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 0x7fbd3814ebe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c7a0 tx=0x7fbd3000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.451+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd30018070 con 0x7fbd381561f0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.451+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd3000a5e0 con 0x7fbd381561f0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.451+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd380acfd0 con 0x7fbd381561f0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.451+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3000aa50 con 0x7fbd381561f0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.451+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd380ad590 con 0x7fbd381561f0 2024-05-09T07:13:39.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.452+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd380756c0 con 0x7fbd381561f0 2024-05-09T07:13:39.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.452+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbd3001b070 con 0x7fbd381561f0 2024-05-09T07:13:39.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.453+0000 7fbd357fa640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbd1806be60 0x7fbd1806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:39.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.453+0000 7fbd37fff640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbd1806be60 0x7fbd1806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:39.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.453+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fbd3008a020 con 0x7fbd381561f0 2024-05-09T07:13:39.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.453+0000 7fbd37fff640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbd1806be60 0x7fbd1806e320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006fd0 tx=0x7fbd28008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:39.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.457+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbd30058db0 con 0x7fbd381561f0 2024-05-09T07:13:39.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.665+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd380ad760 con 0x7fbd381561f0 2024-05-09T07:13:39.667 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.666+0000 7fbd357fa640 1 -- 172.21.15.160:0/1906601604 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1392 (secure 0 0 0) 0x7fbd300586b0 con 0x7fbd381561f0 2024-05-09T07:13:39.668 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:39.668 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-09T07:13:39.668 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 2 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbd1806be60 msgr2=0x7fbd1806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbd1806be60 0x7fbd1806e320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fbd28006fd0 tx=0x7fbd28008040 comp rx=0 tx=0).stop 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 msgr2=0x7fbd3814ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 0x7fbd3814ebe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbd3000c7a0 tx=0x7fbd3000cc60 comp rx=0 tx=0).stop 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 shutdown_connections 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbd1806be60 0x7fbd1806e320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd381561f0 0x7fbd3814ebe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbd380c5f20 0x7fbd3806e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.670 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 --2- 172.21.15.160:0/1906601604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbd3806b6a0 0x7fbd38159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:39.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 >> 172.21.15.160:0/1906601604 conn(0x7fbd380a9800 msgr2=0x7fbd38152480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:39.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 shutdown_connections 2024-05-09T07:13:39.671 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:39.668+0000 7fbd3e75a640 1 -- 172.21.15.160:0/1906601604 wait complete. 2024-05-09T07:13:41.681 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:13:41.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.792+0000 7f299f25d640 1 Processor -- start 2024-05-09T07:13:41.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.792+0000 7f299f25d640 1 -- start start 2024-05-09T07:13:41.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f299f25d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 0x7f2998072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f299f25d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980c25e0 con 0x7f2998152a10 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f299f25d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f29980c28c0 con 0x7f299806e340 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f299f25d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2998073300 con 0x7f29980c33c0 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f299cff9640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2998152a10 0x7f2998071820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51298/0 (socket says 172.21.15.160:51298) 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f299cff9640 1 -- 172.21.15.160:0/1216806486 learned_addr learned my addr 172.21.15.160:0/1216806486 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f2997fff640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 0x7f2998072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.793+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3271036596 0 0) 0x7f29980c25e0 con 0x7f2998152a10 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2980003520 con 0x7f2998152a10 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2710981910 0 0) 0x7f29980c28c0 con 0x7f299806e340 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f29980c25e0 con 0x7f299806e340 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3277064273 0 0) 0x7f2980003520 con 0x7f2998152a10 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 msgr2=0x7f2998072dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:41.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 0x7f2998072dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 >> v1:172.21.15.188:6789/0 conn(0x7f299806e340 legacy=0x7f29980c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29980c2ba0 con 0x7f2998152a10 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f29880015f0 con 0x7f2998152a10 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 >> v1:172.21.15.160:6789/0 conn(0x7f2998152a10 legacy=0x7f2998071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2980003b70 0x7f2980003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 0x7f2980004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f29800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f298000ba70 con 0x7f2980004d40 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f298000bbf0 con 0x7f2980003b70 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f298000bdf0 con 0x7f29980c33c0 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299d7fa640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f29800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f2997fff640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 0x7f2980004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299cff9640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2980003b70 0x7f2980003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299d7fa640 1 -- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 msgr2=0x7f2980004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:41.799 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299d7fa640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980c33c0 0x7f2980004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299d7fa640 1 -- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2980003b70 msgr2=0x7f2980003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299d7fa640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2980003b70 0x7f2980003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.794+0000 7f299d7fa640 1 -- 172.21.15.160:0/1216806486 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2980003520 con 0x7f2980004d40 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.795+0000 7f299d7fa640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f29800051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2990001fd0 tx=0x7f2980005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f696d32475c3309 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.795+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990001bf0 con 0x7f2980004d40 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.795+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2990003b70 con 0x7f2980004d40 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.795+0000 7f29977fe640 1 -- 172.21.15.160:0/1216806486 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29900050d0 con 0x7f2980004d40 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299f25d640 1 -- 172.21.15.160:0/1216806486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 msgr2=0x7f29800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299f25d640 1 --2- 172.21.15.160:0/1216806486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f29800051c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2990001fd0 tx=0x7f2980005830 comp rx=0 tx=0).stop 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299d7fa640 1 -- 172.21.15.160:0/1216806486 reap_dead start 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299f25d640 1 -- 172.21.15.160:0/1216806486 shutdown_connections 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299f25d640 1 -- 172.21.15.160:0/1216806486 >> 172.21.15.160:0/1216806486 conn(0x7f299806b550 msgr2=0x7f299806b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299f25d640 1 -- 172.21.15.160:0/1216806486 shutdown_connections 2024-05-09T07:13:41.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.796+0000 7f299f25d640 1 -- 172.21.15.160:0/1216806486 wait complete. 2024-05-09T07:13:41.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.797+0000 7f299f25d640 1 Processor -- start 2024-05-09T07:13:41.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.797+0000 7f299f25d640 1 -- start start 2024-05-09T07:13:41.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f2998144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980a9870 0x7f29980c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2998155e30 0x7f29981eed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2998071f30 con 0x7f2980004d40 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2998152df0 con 0x7f2998155e30 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f29980c4500 con 0x7f29980a9870 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f2998144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f2998144290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43416/0 (socket says 172.21.15.160:43416) 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 -- 172.21.15.160:0/696990903 learned_addr learned my addr 172.21.15.160:0/696990903 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299d7fa640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2998155e30 0x7f29981eed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f2997fff640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980a9870 0x7f29980c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 -- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980a9870 msgr2=0x7f29980c6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980a9870 0x7f29980c6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 -- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2998155e30 msgr2=0x7f29981eed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2998155e30 0x7f29981eed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 -- 172.21.15.160:0/696990903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299814cd30 con 0x7f2980004d40 2024-05-09T07:13:41.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299cff9640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f2998144290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2988004ef0 tx=0x7f2988007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800f750 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2988018070 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2998144e10 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.798+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298800faf0 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.799+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299814f0a0 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.799+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2998070e30 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.800+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f298800fc90 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.800+0000 7f29957fa640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f298406be30 0x7f298406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.801+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f298808aa20 con 0x7f2980004d40 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.801+0000 7f2997fff640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f298406be30 0x7f298406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:41.805 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.801+0000 7f2997fff640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f298406be30 0x7f298406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29981550a0 tx=0x7f298c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:41.806 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:41.805+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2998070e30 con 0x7f2980004d40 2024-05-09T07:13:42.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.012+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29980756c0 con 0x7f2980004d40 2024-05-09T07:13:42.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.012+0000 7f29957fa640 1 -- 172.21.15.160:0/696990903 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f29880597b0 con 0x7f2980004d40 2024-05-09T07:13:42.014 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:42.014 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T07:13:42.014 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 3 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f298406be30 msgr2=0x7f298406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f298406be30 0x7f298406e2f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29981550a0 tx=0x7f298c008040 comp rx=0 tx=0).stop 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 msgr2=0x7f2998144290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f2998144290 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2988004ef0 tx=0x7f2988007430 comp rx=0 tx=0).stop 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 shutdown_connections 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f298406be30 0x7f298406e2f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2998155e30 0x7f29981eed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f29980a9870 0x7f29980c6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 --2- 172.21.15.160:0/696990903 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2980004d40 0x7f2998144290 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 >> 172.21.15.160:0/696990903 conn(0x7f299806b550 msgr2=0x7f29981532a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 shutdown_connections 2024-05-09T07:13:42.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.014+0000 7f299f25d640 1 -- 172.21.15.160:0/696990903 wait complete. 2024-05-09T07:13:42.027 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:42.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 Processor -- start 2024-05-09T07:13:42.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 -- start start 2024-05-09T07:13:42.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24b80be320 0x7f24b8155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f24b8159840 con 0x7f24b80c8b90 2024-05-09T07:13:42.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f24b8159310 con 0x7f24b80be320 2024-05-09T07:13:42.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.126+0000 7f24bec2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f24b8159560 con 0x7f24b80c5610 2024-05-09T07:13:42.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24b7fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f24b80be320 0x7f24b8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24bd1c9640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f24b80c8b90 0x7f24b8155850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51306/0 (socket says 172.21.15.160:51306) 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24bd1c9640 1 -- 172.21.15.160:0/2779979505 learned_addr learned my addr 172.21.15.160:0/2779979505 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24bc9c8640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24b7fff640 1 -- 172.21.15.160:0/2779979505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 msgr2=0x7f24b80c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24b7fff640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24b77fe640 1 -- 172.21.15.160:0/2779979505 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2893820370 0 0) 0x7f24b8159840 con 0x7f24b80c8b90 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24b7fff640 1 -- 172.21.15.160:0/2779979505 >> v1:172.21.15.160:6789/0 conn(0x7f24b80c8b90 legacy=0x7f24b8155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24b7fff640 1 -- 172.21.15.160:0/2779979505 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b80c77d0 con 0x7f24b80be320 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.127+0000 7f24bc9c8640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24b7fff640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8155310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f24a40097a0 tx=0x7f24a402a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f281ac6c74c569b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24b77fe640 1 -- 172.21.15.160:0/2779979505 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4037070 con 0x7f24b80be320 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24b77fe640 1 -- 172.21.15.160:0/2779979505 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24a402adb0 con 0x7f24b80be320 2024-05-09T07:13:42.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24b77fe640 1 -- 172.21.15.160:0/2779979505 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24a4032660 con 0x7f24b80be320 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 -- 172.21.15.160:0/2779979505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 msgr2=0x7f24b8155310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8155310 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f24a40097a0 tx=0x7f24a402a060 comp rx=0 tx=0).stop 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 -- 172.21.15.160:0/2779979505 shutdown_connections 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8155310 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 --2- 172.21.15.160:0/2779979505 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 -- 172.21.15.160:0/2779979505 >> 172.21.15.160:0/2779979505 conn(0x7f24b8072e90 msgr2=0x7f24b80ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 -- 172.21.15.160:0/2779979505 shutdown_connections 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.128+0000 7f24bec2c640 1 -- 172.21.15.160:0/2779979505 wait complete. 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.129+0000 7f24bec2c640 1 Processor -- start 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.129+0000 7f24bec2c640 1 -- start start 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bec2c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bec2c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bec2c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24b80c8b90 0x7f24b80c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bec2c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f24b80ad6a0 con 0x7f24b80be320 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bec2c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f24b80c7d90 con 0x7f24b80c8b90 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bec2c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f24b8155f60 con 0x7f24b80c5610 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bc9c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bc9c8640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43440/0 (socket says 172.21.15.160:43440) 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.130+0000 7f24bc9c8640 1 -- 172.21.15.160:0/455114095 learned_addr learned my addr 172.21.15.160:0/455114095 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bd1c9640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24b80c8b90 0x7f24b80c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24b7fff640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bc9c8640 1 -- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 msgr2=0x7f24b80bff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bc9c8640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80bff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bc9c8640 1 -- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24b80c8b90 msgr2=0x7f24b80c04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bc9c8640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24b80c8b90 0x7f24b80c04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bc9c8640 1 -- 172.21.15.160:0/455114095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24b81eaa60 con 0x7f24b80be320 2024-05-09T07:13:42.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bc9c8640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8158250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c7f0 tx=0x7f24ac00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac015070 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f24ac007a30 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24b81ead20 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24ac007e00 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.131+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24b81eb2e0 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.132+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24b80756c0 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.132+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f24ac018070 con 0x7f24b80be320 2024-05-09T07:13:42.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.133+0000 7f24b57fa640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.133+0000 7f24b7fff640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.133+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f24ac08aad0 con 0x7f24b80be320 2024-05-09T07:13:42.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.133+0000 7f24b7fff640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f249c06be30 0x7f249c06e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24a4034000 tx=0x7f24a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.137+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f24ac0597d0 con 0x7f24b80be320 2024-05-09T07:13:42.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.312+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f24b80ac5c0 con 0x7f24b80be320 2024-05-09T07:13:42.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.312+0000 7f24b57fa640 1 -- 172.21.15.160:0/455114095 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) ==== 74+0+12920 (secure 0 0 0) 0x7f24ac0590d0 con 0x7f24b80be320 2024-05-09T07:13:42.314 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:42.314 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":16,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:35.358666+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f249c06be30 msgr2=0x7f249c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f249c06be30 0x7f249c06e2f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f24a4034000 tx=0x7f24a402e000 comp rx=0 tx=0).stop 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 msgr2=0x7f24b8158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8158250 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f24ac00c7f0 tx=0x7f24ac00cbf0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 shutdown_connections 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f249c06be30 0x7f249c06e2f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.314+0000 7f24bec2c640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f24b80c8b90 0x7f24b80c04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.315+0000 7f24bec2c640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f24b80c5610 0x7f24b80bff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.315+0000 7f24bec2c640 1 --2- 172.21.15.160:0/455114095 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b80be320 0x7f24b8158250 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.315+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 >> 172.21.15.160:0/455114095 conn(0x7f24b8072e90 msgr2=0x7f24b814e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.315+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 shutdown_connections 2024-05-09T07:13:42.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.315+0000 7f24bec2c640 1 -- 172.21.15.160:0/455114095 wait complete. 2024-05-09T07:13:42.327 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T07:13:42.328 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:13:42.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.430+0000 7f4c7ebf9640 1 Processor -- start 2024-05-09T07:13:42.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.430+0000 7f4c7ebf9640 1 -- start start 2024-05-09T07:13:42.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7ebf9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4c780719e0 0x7f4c780c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7ebf9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c781595e0 con 0x7f4c78071f80 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7ebf9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c781598c0 con 0x7f4c7806b160 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7ebf9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f4c780730a0 con 0x7f4c780719e0 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7d196640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4c780719e0 0x7f4c780c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c77fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f4c78071f80 0x7f4c7815deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51308/0 (socket says 172.21.15.160:51308) 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c77fff640 1 -- 172.21.15.160:0/1233547731 learned_addr learned my addr 172.21.15.160:0/1233547731 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7d196640 1 -- 172.21.15.160:0/1233547731 >> v1:172.21.15.188:6789/0 conn(0x7f4c7806b160 legacy=0x7f4c78155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:42.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7d196640 1 -- 172.21.15.160:0/1233547731 >> v1:172.21.15.160:6789/0 conn(0x7f4c78071f80 legacy=0x7f4c7815deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.431+0000 7f4c7d196640 1 -- 172.21.15.160:0/1233547731 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c780732f0 con 0x7f4c780719e0 2024-05-09T07:13:42.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7d196640 1 --2- 172.21.15.160:0/1233547731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c780719e0 0x7f4c780c6180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009d90 tx=0x7f4c7002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faddc788013b611 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c777fe640 1 -- 172.21.15.160:0/1233547731 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70007740 con 0x7f4c780719e0 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c777fe640 1 -- 172.21.15.160:0/1233547731 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c700078e0 con 0x7f4c780719e0 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c777fe640 1 -- 172.21.15.160:0/1233547731 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70007cb0 con 0x7f4c780719e0 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/1233547731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c780719e0 msgr2=0x7f4c780c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/1233547731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c780719e0 0x7f4c780c6180 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4c70009d90 tx=0x7f4c7002a580 comp rx=0 tx=0).stop 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/1233547731 shutdown_connections 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/1233547731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c780719e0 0x7f4c780c6180 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/1233547731 >> 172.21.15.160:0/1233547731 conn(0x7f4c780a98a0 msgr2=0x7f4c780a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/1233547731 shutdown_connections 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.432+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/1233547731 wait complete. 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.433+0000 7f4c7ebf9640 1 Processor -- start 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 -- start start 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 0x7f4c7814bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c780719e0 0x7f4c7806d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c78071f80 0x7f4c781e0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4c78152e30 con 0x7f4c7806b160 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4c78073660 con 0x7f4c780719e0 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7ebf9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4c78152fb0 con 0x7f4c78071f80 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7c995640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 0x7f4c7814bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7c995640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 0x7f4c7814bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43458/0 (socket says 172.21.15.160:43458) 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.434+0000 7f4c7c995640 1 -- 172.21.15.160:0/411404712 learned_addr learned my addr 172.21.15.160:0/411404712 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c77fff640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c780719e0 0x7f4c7806d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7d196640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c78071f80 0x7f4c781e0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7c995640 1 -- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c78071f80 msgr2=0x7f4c781e0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7c995640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c78071f80 0x7f4c781e0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7c995640 1 -- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c780719e0 msgr2=0x7f4c7806d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7c995640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c780719e0 0x7f4c7806d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7c995640 1 -- 172.21.15.160:0/411404712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c781dfba0 con 0x7f4c7806b160 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c77fff640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c780719e0 0x7f4c7806d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:42.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7c995640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 0x7f4c7814bf70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b3f0 tx=0x7f4c6c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c015020 con 0x7f4c7806b160 2024-05-09T07:13:42.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c6c011830 con 0x7f4c7806b160 2024-05-09T07:13:42.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c781576a0 con 0x7f4c7806b160 2024-05-09T07:13:42.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6c011c20 con 0x7f4c7806b160 2024-05-09T07:13:42.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.435+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c78157c60 con 0x7f4c7806b160 2024-05-09T07:13:42.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.436+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c78075650 con 0x7f4c7806b160 2024-05-09T07:13:42.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.436+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4c6c018020 con 0x7f4c7806b160 2024-05-09T07:13:42.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.437+0000 7f4c757fa640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4c5c06be30 0x7f4c5c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.437+0000 7f4c77fff640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4c5c06be30 0x7f4c5c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.437+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f4c6c08a720 con 0x7f4c7806b160 2024-05-09T07:13:42.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.437+0000 7f4c77fff640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4c5c06be30 0x7f4c5c06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c78074be0 tx=0x7f4c64009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.441+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c6c01f030 con 0x7f4c7806b160 2024-05-09T07:13:42.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.616+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c7806be80 con 0x7f4c7806b160 2024-05-09T07:13:42.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.617+0000 7f4c757fa640 1 -- 172.21.15.160:0/411404712 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) ==== 93+0+10 (secure 0 0 0) 0x7f4c6c0172c0 con 0x7f4c7806b160 2024-05-09T07:13:42.618 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:13:42.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.618+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4c5c06be30 msgr2=0x7f4c5c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.618+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4c5c06be30 0x7f4c5c06e2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4c78074be0 tx=0x7f4c64009040 comp rx=0 tx=0).stop 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.618+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 msgr2=0x7f4c7814bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.618+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 0x7f4c7814bf70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c00b3f0 tx=0x7f4c6c00b8b0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 shutdown_connections 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4c5c06be30 0x7f4c5c06e2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4c780719e0 0x7f4c7806d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4c78071f80 0x7f4c781e0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 --2- 172.21.15.160:0/411404712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4c7806b160 0x7f4c7814bf70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 >> 172.21.15.160:0/411404712 conn(0x7f4c780a98a0 msgr2=0x7f4c780a8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 shutdown_connections 2024-05-09T07:13:42.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.619+0000 7f4c7ebf9640 1 -- 172.21.15.160:0/411404712 wait complete. 2024-05-09T07:13:42.631 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-05-09T07:13:42.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.730+0000 7fe4b09c1640 1 Processor -- start 2024-05-09T07:13:42.732 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.730+0000 7fe4b09c1640 1 -- start start 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4b09c1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe4ac155f50 0x7fe4ac160d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4b09c1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ac159100 con 0x7fe4ac073210 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4b09c1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4ac0a7210 con 0x7fe4ac0c59c0 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4b09c1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe4ac0c2000 con 0x7fe4ac155f50 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4aa59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe4ac073210 0x7fe4ac1589f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35278/0 (socket says 172.21.15.160:35278) 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4aa59c640 1 -- 172.21.15.160:0/3505939422 learned_addr learned my addr 172.21.15.160:0/3505939422 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4aad9d640 1 --2- 172.21.15.160:0/3505939422 >> v2:172.21.15.160:3300/0 conn(0x7fe4ac155f50 0x7fe4ac160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4a959a640 1 -- 172.21.15.160:0/3505939422 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3277057630 0 0) 0x7fe4ac159100 con 0x7fe4ac073210 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4a959a640 1 -- 172.21.15.160:0/3505939422 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe494003520 con 0x7fe4ac073210 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4aad9d640 1 -- 172.21.15.160:0/3505939422 >> v1:172.21.15.188:6789/0 conn(0x7fe4ac0c59c0 legacy=0x7fe4ac06c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4aad9d640 1 -- 172.21.15.160:0/3505939422 >> v1:172.21.15.160:6789/0 conn(0x7fe4ac073210 legacy=0x7fe4ac1589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.731+0000 7fe4aad9d640 1 -- 172.21.15.160:0/3505939422 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac0a74f0 con 0x7fe4ac155f50 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.732+0000 7fe4aad9d640 1 --2- 172.21.15.160:0/3505939422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac155f50 0x7fe4ac160d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009d90 tx=0x7fe4a402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65656ac834f9b84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.732+0000 7fe4a959a640 1 -- 172.21.15.160:0/3505939422 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4007740 con 0x7fe4ac155f50 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.732+0000 7fe4a959a640 1 -- 172.21.15.160:0/3505939422 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4a40078e0 con 0x7fe4ac155f50 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.732+0000 7fe4a959a640 1 -- 172.21.15.160:0/3505939422 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a4007cb0 con 0x7fe4ac155f50 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 -- 172.21.15.160:0/3505939422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac155f50 msgr2=0x7fe4ac160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/3505939422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac155f50 0x7fe4ac160d90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe4a4009d90 tx=0x7fe4a402a6f0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 -- 172.21.15.160:0/3505939422 shutdown_connections 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/3505939422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac155f50 0x7fe4ac160d90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 -- 172.21.15.160:0/3505939422 >> 172.21.15.160:0/3505939422 conn(0x7fe4ac0a9c90 msgr2=0x7fe4ac0aa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 -- 172.21.15.160:0/3505939422 shutdown_connections 2024-05-09T07:13:42.735 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.733+0000 7fe4b09c1640 1 -- 172.21.15.160:0/3505939422 wait complete. 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.734+0000 7fe4b09c1640 1 Processor -- start 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.734+0000 7fe4b09c1640 1 -- start start 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 0x7fe4ac0abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4ac0c59c0 0x7fe4ac0acbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4ac155f50 0x7fe4ac153da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac155400 con 0x7fe4ac073210 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4ac155a30 con 0x7fe4ac0c59c0 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe4ac0735f0 con 0x7fe4ac155f50 2024-05-09T07:13:42.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 0x7fe4ac0abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 0x7fe4ac0abc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40024/0 (socket says 172.21.15.160:40024) 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 -- 172.21.15.160:0/1002645961 learned_addr learned my addr 172.21.15.160:0/1002645961 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4a9d9b640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4ac0c59c0 0x7fe4ac0acbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aad9d640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4ac155f50 0x7fe4ac153da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 -- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4ac155f50 msgr2=0x7fe4ac153da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4ac155f50 0x7fe4ac153da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 -- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4ac0c59c0 msgr2=0x7fe4ac0acbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4ac0c59c0 0x7fe4ac0acbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 -- 172.21.15.160:0/1002645961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4ac0aab40 con 0x7fe4ac073210 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4a9d9b640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4ac0c59c0 0x7fe4ac0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4aa59c640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 0x7fe4ac0abc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c500 tx=0x7fe49c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c017070 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49c005ce0 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4ac0ad8f0 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe49c007330 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.735+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4ac0ad100 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.737+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4ac0756c0 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.737+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe49c01a070 con 0x7fe4ac073210 2024-05-09T07:13:42.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.737+0000 7fe49affd640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe48c06be30 0x7fe48c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:42.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.737+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7fe49c089f30 con 0x7fe4ac073210 2024-05-09T07:13:42.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.737+0000 7fe4a9d9b640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe48c06be30 0x7fe48c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:42.741 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.738+0000 7fe4a9d9b640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe48c06be30 0x7fe48c06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac1ebe60 tx=0x7fe4a0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:42.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.741+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe49c058d80 con 0x7fe4ac073210 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.916+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe4ac1542e0 con 0x7fe4ac073210 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.916+0000 7fe49affd640 1 -- 172.21.15.160:0/1002645961 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) ==== 84+0+65 (secure 0 0 0) 0x7fe49c058680 con 0x7fe4ac073210 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stdout:[ 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stdout: { 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stdout: "poolnum": 1, 2024-05-09T07:13:42.918 INFO:teuthology.orchestra.run.smithi160.stdout: "poolname": ".mgr" 2024-05-09T07:13:42.919 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T07:13:42.919 INFO:teuthology.orchestra.run.smithi160.stdout:] 2024-05-09T07:13:42.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe48c06be30 msgr2=0x7fe48c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe48c06be30 0x7fe48c06e2f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac1ebe60 tx=0x7fe4a0009040 comp rx=0 tx=0).stop 2024-05-09T07:13:42.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 msgr2=0x7fe4ac0abc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:42.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 0x7fe4ac0abc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe49c00c500 tx=0x7fe49c00c9c0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 shutdown_connections 2024-05-09T07:13:42.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe48c06be30 0x7fe48c06e2f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4ac0c59c0 0x7fe4ac0acbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4ac155f50 0x7fe4ac153da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 --2- 172.21.15.160:0/1002645961 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4ac073210 0x7fe4ac0abc80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:42.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 >> 172.21.15.160:0/1002645961 conn(0x7fe4ac0a9c90 msgr2=0x7fe4ac06c660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:42.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 shutdown_connections 2024-05-09T07:13:42.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:42.918+0000 7fe4b09c1640 1 -- 172.21.15.160:0/1002645961 wait complete. 2024-05-09T07:13:42.931 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T07:13:42.931 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T07:13:43.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.033+0000 7f2d31888640 1 Processor -- start 2024-05-09T07:13:43.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.033+0000 7f2d31888640 1 -- start start 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d31888640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c0c2390 0x7f2d2c0c27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d31888640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d2c15dea0 0x7f2d2c160290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d31888640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d2c0c3570 con 0x7f2d2c152b60 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d31888640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f2d2c0c1f00 con 0x7f2d2c15dea0 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d31888640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0c64c0 con 0x7f2d2c0c2390 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d2b7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d2c15dea0 0x7f2d2c160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d2b7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2d2c15dea0 0x7f2d2c160290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40026/0 (socket says 172.21.15.160:40026) 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d2b7fe640 1 -- 172.21.15.160:0/358402240 learned_addr learned my addr 172.21.15.160:0/358402240 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d2a7fc640 1 --2- 172.21.15.160:0/358402240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c0c2390 0x7f2d2c0c27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d2b7fe640 1 -- 172.21.15.160:0/358402240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c0c2390 msgr2=0x7f2d2c0c27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:43.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.034+0000 7f2d2b7fe640 1 --2- 172.21.15.160:0/358402240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c0c2390 0x7f2d2c0c27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.035+0000 7f2d2b7fe640 1 -- 172.21.15.160:0/358402240 >> v1:172.21.15.188:6789/0 conn(0x7f2d2c152b60 legacy=0x7f2d2c0c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.035+0000 7f2d2b7fe640 1 -- 172.21.15.160:0/358402240 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c0c37e0 con 0x7f2d2c15dea0 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.035+0000 7f2d2b7fe640 1 --2- 172.21.15.160:0/358402240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c15dea0 0x7f2d2c160290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009e50 tx=0x7f2d2402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd364ba2fa6a1bfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.035+0000 7f2d29ffb640 1 -- 172.21.15.160:0/358402240 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d24036070 con 0x7f2d2c15dea0 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.035+0000 7f2d29ffb640 1 -- 172.21.15.160:0/358402240 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d24031600 con 0x7f2d2c15dea0 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.035+0000 7f2d29ffb640 1 -- 172.21.15.160:0/358402240 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d240319d0 con 0x7f2d2c15dea0 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 -- 172.21.15.160:0/358402240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c15dea0 msgr2=0x7f2d2c160290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 --2- 172.21.15.160:0/358402240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c15dea0 0x7f2d2c160290 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2d24009e50 tx=0x7f2d2402a680 comp rx=0 tx=0).stop 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 -- 172.21.15.160:0/358402240 shutdown_connections 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 --2- 172.21.15.160:0/358402240 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c15dea0 0x7f2d2c160290 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 --2- 172.21.15.160:0/358402240 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c0c2390 0x7f2d2c0c27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 -- 172.21.15.160:0/358402240 >> 172.21.15.160:0/358402240 conn(0x7f2d2c0c8ad0 msgr2=0x7f2d2c0c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:43.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 -- 172.21.15.160:0/358402240 shutdown_connections 2024-05-09T07:13:43.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.036+0000 7f2d31888640 1 -- 172.21.15.160:0/358402240 wait complete. 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.037+0000 7f2d31888640 1 Processor -- start 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.037+0000 7f2d31888640 1 -- start start 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d31888640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d2c0c2390 0x7f2d2c1ee8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d31888640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 0x7f2d2c0ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d31888640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c15dea0 0x7f2d2c0acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d31888640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c156420 con 0x7f2d2c152b60 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d31888640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2c152f40 con 0x7f2d2c0c2390 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d31888640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2d2c0c5c70 con 0x7f2d2c15dea0 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 0x7f2d2c0ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 0x7f2d2c0ac610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40038/0 (socket says 172.21.15.160:40038) 2024-05-09T07:13:43.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 -- 172.21.15.160:0/1115426601 learned_addr learned my addr 172.21.15.160:0/1115426601 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2b7fe640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c15dea0 0x7f2d2c0acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2affd640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d2c0c2390 0x7f2d2c1ee8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 -- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c15dea0 msgr2=0x7f2d2c0acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c15dea0 0x7f2d2c0acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 -- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d2c0c2390 msgr2=0x7f2d2c1ee8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d2c0c2390 0x7f2d2c1ee8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 -- 172.21.15.160:0/1115426601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2c152260 con 0x7f2d2c152b60 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.038+0000 7f2d2a7fc640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 0x7f2d2c0ac610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d830 tx=0x7f2d2000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.039+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000b6c0 con 0x7f2d2c152b60 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.039+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d20019070 con 0x7f2d2c152b60 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.039+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2c1ef2e0 con 0x7f2d2c152b60 2024-05-09T07:13:43.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.039+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000ba80 con 0x7f2d2c152b60 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.039+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d2c06cff0 con 0x7f2d2c152b60 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.040+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d2c0c4d90 con 0x7f2d2c152b60 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.040+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2d2000bc20 con 0x7f2d2c152b60 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.040+0000 7f2d13fff640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2d1806be30 0x7f2d1806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.041+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(16..16 src has 1..16) ==== 4857+0+0 (secure 0 0 0) 0x7f2d2008af50 con 0x7f2d2c152b60 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.041+0000 7f2d2affd640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2d1806be30 0x7f2d1806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.041+0000 7f2d2affd640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2d1806be30 0x7f2d1806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c159b40 tx=0x7f2d1c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:43.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.045+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2d2001c360 con 0x7f2d2c152b60 2024-05-09T07:13:43.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.219+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2d2c0c61c0 con 0x7f2d2c152b60 2024-05-09T07:13:43.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.447+0000 7f2d13fff640 1 -- 172.21.15.160:0/1115426601 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) ==== 138+0+0 (secure 0 0 0) 0x7f2d20059ce0 con 0x7f2d2c152b60 2024-05-09T07:13:43.450 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_metadata' created 2024-05-09T07:13:43.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2d1806be30 msgr2=0x7f2d1806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2d1806be30 0x7f2d1806e2f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2d2c159b40 tx=0x7f2d1c009040 comp rx=0 tx=0).stop 2024-05-09T07:13:43.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 msgr2=0x7f2d2c0ac610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 0x7f2d2c0ac610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2d2000d830 tx=0x7f2d2000dcf0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 shutdown_connections 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2d1806be30 0x7f2d1806e2f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2d2c15dea0 0x7f2d2c0acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d2c152b60 0x7f2d2c0ac610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 --2- 172.21.15.160:0/1115426601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2d2c0c2390 0x7f2d2c1ee8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.449+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 >> 172.21.15.160:0/1115426601 conn(0x7f2d2c0c8ad0 msgr2=0x7f2d2c074270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.450+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 shutdown_connections 2024-05-09T07:13:43.452 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.450+0000 7f2d31888640 1 -- 172.21.15.160:0/1115426601 wait complete. 2024-05-09T07:13:43.467 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-09T07:13:43.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.571+0000 7f9f69029640 1 Processor -- start 2024-05-09T07:13:43.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.571+0000 7f9f69029640 1 -- start start 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f69029640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9f640c5d80 0x7f9f640c91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f69029640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6406c5e0 0x7f9f6406ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f69029640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9f64158c30 con 0x7f9f640c8e10 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f69029640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f9f64158f10 con 0x7f9f640c5d80 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f69029640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f641591f0 con 0x7f9f6406c5e0 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f6259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9f640c5d80 0x7f9f640c91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f6259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9f640c5d80 0x7f9f640c91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40050/0 (socket says 172.21.15.160:40050) 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.572+0000 7f9f6259c640 1 -- 172.21.15.160:0/3446648680 learned_addr learned my addr 172.21.15.160:0/3446648680 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.573+0000 7f9f6259c640 1 -- 172.21.15.160:0/3446648680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6406c5e0 msgr2=0x7f9f6406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.573+0000 7f9f6359e640 1 --2- 172.21.15.160:0/3446648680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6406c5e0 0x7f9f6406ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.573+0000 7f9f6259c640 1 --2- 172.21.15.160:0/3446648680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6406c5e0 0x7f9f6406ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.573+0000 7f9f6259c640 1 -- 172.21.15.160:0/3446648680 >> v1:172.21.15.188:6789/0 conn(0x7f9f640c8e10 legacy=0x7f9f640c5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.573+0000 7f9f6259c640 1 -- 172.21.15.160:0/3446648680 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f641594d0 con 0x7f9f640c5d80 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.573+0000 7f9f6259c640 1 --2- 172.21.15.160:0/3446648680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c5d80 0x7f9f640c91f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009870 tx=0x7f9f5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3607a8bd899079ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f61d9b640 1 -- 172.21.15.160:0/3446648680 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58037070 con 0x7f9f640c5d80 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f61d9b640 1 -- 172.21.15.160:0/3446648680 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f58032440 con 0x7f9f640c5d80 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f61d9b640 1 -- 172.21.15.160:0/3446648680 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f58032810 con 0x7f9f640c5d80 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 -- 172.21.15.160:0/3446648680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c5d80 msgr2=0x7f9f640c91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 --2- 172.21.15.160:0/3446648680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c5d80 0x7f9f640c91f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009870 tx=0x7f9f5802a180 comp rx=0 tx=0).stop 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 -- 172.21.15.160:0/3446648680 shutdown_connections 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 --2- 172.21.15.160:0/3446648680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c5d80 0x7f9f640c91f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 --2- 172.21.15.160:0/3446648680 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f6406c5e0 0x7f9f6406ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 -- 172.21.15.160:0/3446648680 >> 172.21.15.160:0/3446648680 conn(0x7f9f640aca60 msgr2=0x7f9f640ace70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 -- 172.21.15.160:0/3446648680 shutdown_connections 2024-05-09T07:13:43.577 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.574+0000 7f9f69029640 1 -- 172.21.15.160:0/3446648680 wait complete. 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 Processor -- start 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 -- start start 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f6406c5e0 0x7f9f64155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f640c5d80 0x7f9f640c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 0x7f9f64149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64152f00 con 0x7f9f640c8e10 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f64152d80 con 0x7f9f6406c5e0 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f69029640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f6406cf70 con 0x7f9f640c5d80 2024-05-09T07:13:43.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f6359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 0x7f9f64149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f6359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 0x7f9f64149aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40056/0 (socket says 172.21.15.160:40056) 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.576+0000 7f9f6359e640 1 -- 172.21.15.160:0/2169830674 learned_addr learned my addr 172.21.15.160:0/2169830674 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6259c640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f640c5d80 0x7f9f640c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f62d9d640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f6406c5e0 0x7f9f64155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6359e640 1 -- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f640c5d80 msgr2=0x7f9f640c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6359e640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f640c5d80 0x7f9f640c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6359e640 1 -- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f6406c5e0 msgr2=0x7f9f64155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6359e640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f6406c5e0 0x7f9f64155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6359e640 1 -- 172.21.15.160:0/2169830674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f64157c30 con 0x7f9f640c8e10 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6259c640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f640c5d80 0x7f9f640c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f62d9d640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f6406c5e0 0x7f9f64155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f6359e640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 0x7f9f64149aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cd70 tx=0x7f9f5c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c003b90 con 0x7f9f640c8e10 2024-05-09T07:13:43.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f5c003d30 con 0x7f9f640c8e10 2024-05-09T07:13:43.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c004240 con 0x7f9f640c8e10 2024-05-09T07:13:43.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f64153330 con 0x7f9f640c8e10 2024-05-09T07:13:43.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.577+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f641538b0 con 0x7f9f640c8e10 2024-05-09T07:13:43.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.578+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f640756c0 con 0x7f9f640c8e10 2024-05-09T07:13:43.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.579+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f5c0043e0 con 0x7f9f640c8e10 2024-05-09T07:13:43.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.579+0000 7f9f4b7fe640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f5006be30 0x7f9f5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:43.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.579+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 5227+0+0 (secure 0 0 0) 0x7f9f5c08b6f0 con 0x7f9f640c8e10 2024-05-09T07:13:43.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.579+0000 7f9f62d9d640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f5006be30 0x7f9f5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:43.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.580+0000 7f9f62d9d640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f5006be30 0x7f9f5006e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f54006fd0 tx=0x7f9f54008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:43.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.584+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f5c05a310 con 0x7f9f640c8e10 2024-05-09T07:13:43.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:43.762+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9f640caad0 con 0x7f9f640c8e10 2024-05-09T07:13:44.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.449+0000 7f9f4b7fe640 1 -- 172.21.15.160:0/2169830674 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) ==== 171+0+0 (secure 0 0 0) 0x7f9f5c059c10 con 0x7f9f640c8e10 2024-05-09T07:13:44.451 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_data' created 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f5006be30 msgr2=0x7f9f5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f5006be30 0x7f9f5006e2f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9f54006fd0 tx=0x7f9f54008040 comp rx=0 tx=0).stop 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 msgr2=0x7f9f64149aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 0x7f9f64149aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00cd70 tx=0x7f9f5c00a660 comp rx=0 tx=0).stop 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 shutdown_connections 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f5006be30 0x7f9f5006e2f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f640c8e10 0x7f9f64149aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f640c5d80 0x7f9f640c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 --2- 172.21.15.160:0/2169830674 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f6406c5e0 0x7f9f64155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 >> 172.21.15.160:0/2169830674 conn(0x7f9f640aca60 msgr2=0x7f9f6414c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 shutdown_connections 2024-05-09T07:13:44.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.452+0000 7f9f69029640 1 -- 172.21.15.160:0/2169830674 wait complete. 2024-05-09T07:13:44.476 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T07:13:44.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.577+0000 7ff1ae509640 1 Processor -- start 2024-05-09T07:13:44.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.577+0000 7ff1ae509640 1 -- start start 2024-05-09T07:13:44.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1ae509640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff1a806c3e0 0x7ff1a80c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1ae509640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1a8066f90 0x7ff1a80c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1ae509640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1a80ac3b0 con 0x7ff1a814b990 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1ae509640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a8155f10 con 0x7ff1a8066f90 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1ae509640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff1a80ac650 con 0x7ff1a806c3e0 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1a8066f90 0x7ff1a80c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff1a8066f90 0x7ff1a80c0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40072/0 (socket says 172.21.15.160:40072) 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 -- 172.21.15.160:0/2437994214 learned_addr learned my addr 172.21.15.160:0/2437994214 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 -- 172.21.15.160:0/2437994214 >> v2:172.21.15.188:3300/0 conn(0x7ff1a806c3e0 msgr2=0x7ff1a80c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 --2- 172.21.15.160:0/2437994214 >> v2:172.21.15.188:3300/0 conn(0x7ff1a806c3e0 0x7ff1a80c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 -- 172.21.15.160:0/2437994214 >> v1:172.21.15.188:6789/0 conn(0x7ff1a814b990 legacy=0x7ff1a80ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.578+0000 7ff1a77fe640 1 -- 172.21.15.160:0/2437994214 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a81561f0 con 0x7ff1a8066f90 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.579+0000 7ff1a77fe640 1 --2- 172.21.15.160:0/2437994214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8066f90 0x7ff1a80c0760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8074cd0 tx=0x7ff19c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff157ef3bc316652 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.579+0000 7ff1a6ffd640 1 -- 172.21.15.160:0/2437994214 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c037070 con 0x7ff1a8066f90 2024-05-09T07:13:44.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.579+0000 7ff1a6ffd640 1 -- 172.21.15.160:0/2437994214 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19c032440 con 0x7ff1a8066f90 2024-05-09T07:13:44.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.579+0000 7ff1a6ffd640 1 -- 172.21.15.160:0/2437994214 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19c032810 con 0x7ff1a8066f90 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 -- 172.21.15.160:0/2437994214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8066f90 msgr2=0x7ff1a80c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2437994214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8066f90 0x7ff1a80c0760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8074cd0 tx=0x7ff19c02a180 comp rx=0 tx=0).stop 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 -- 172.21.15.160:0/2437994214 shutdown_connections 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2437994214 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a8066f90 0x7ff1a80c0760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2437994214 >> v2:172.21.15.188:3300/0 conn(0x7ff1a806c3e0 0x7ff1a80c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 -- 172.21.15.160:0/2437994214 >> 172.21.15.160:0/2437994214 conn(0x7ff1a80c3e30 msgr2=0x7ff1a80c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 -- 172.21.15.160:0/2437994214 shutdown_connections 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.580+0000 7ff1ae509640 1 -- 172.21.15.160:0/2437994214 wait complete. 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.581+0000 7ff1ae509640 1 Processor -- start 2024-05-09T07:13:44.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.581+0000 7ff1ae509640 1 -- start start 2024-05-09T07:13:44.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a8066f90 0x7ff1a80aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:44.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a806c3e0 0x7ff1a8146c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:44.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 0x7ff1a8147180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:44.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a80c4d80 con 0x7ff1a814b990 2024-05-09T07:13:44.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a80c4c00 con 0x7ff1a8066f90 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a806c7c0 con 0x7ff1a806c3e0 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 0x7ff1a8147180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 0x7ff1a8147180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40084/0 (socket says 172.21.15.160:40084) 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 -- 172.21.15.160:0/2769952267 learned_addr learned my addr 172.21.15.160:0/2769952267 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1a77fe640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a806c3e0 0x7ff1a8146c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1a7fff640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a8066f90 0x7ff1a80aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 -- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a806c3e0 msgr2=0x7ff1a8146c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a806c3e0 0x7ff1a8146c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 -- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a8066f90 msgr2=0x7ff1a80aa8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a8066f90 0x7ff1a80aa8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 -- 172.21.15.160:0/2769952267 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a81eaf80 con 0x7ff1a814b990 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1acaa6640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 0x7ff1a8147180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000b990 tx=0x7ff1a000bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0019070 con 0x7ff1a814b990 2024-05-09T07:13:44.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1a000caa0 con 0x7ff1a814b990 2024-05-09T07:13:44.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a81eb260 con 0x7ff1a814b990 2024-05-09T07:13:44.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.582+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0014440 con 0x7ff1a814b990 2024-05-09T07:13:44.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.583+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a81eb820 con 0x7ff1a814b990 2024-05-09T07:13:44.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.584+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a806dd30 con 0x7ff1a814b990 2024-05-09T07:13:44.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.584+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff1a001c070 con 0x7ff1a814b990 2024-05-09T07:13:44.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.585+0000 7ff1a4ff9640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff19406be30 0x7ff19406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:44.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.585+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5609+0+0 (secure 0 0 0) 0x7ff1a008a9d0 con 0x7ff1a814b990 2024-05-09T07:13:44.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.585+0000 7ff1a7fff640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff19406be30 0x7ff19406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:44.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.585+0000 7ff1a7fff640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff19406be30 0x7ff19406e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff198009990 tx=0x7ff198009110 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:44.590 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.589+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff1a0058da0 con 0x7ff1a814b990 2024-05-09T07:13:44.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:44.802+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff1a806fb20 con 0x7ff1a814b990 2024-05-09T07:13:45.437 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:13:45.440 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:13:45.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.471+0000 7ff1a4ff9640 1 -- 172.21.15.160:0/2769952267 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7ff1a005d0a0 con 0x7ff1a814b990 2024-05-09T07:13:45.476 INFO:teuthology.orchestra.run.smithi160.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T07:13:45.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.483+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff19406be30 msgr2=0x7ff19406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:45.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.483+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff19406be30 0x7ff19406e2f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff198009990 tx=0x7ff198009110 comp rx=0 tx=0).stop 2024-05-09T07:13:45.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.483+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 msgr2=0x7ff1a8147180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:45.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.483+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 0x7ff1a8147180 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff1a000b990 tx=0x7ff1a000bd90 comp rx=0 tx=0).stop 2024-05-09T07:13:45.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.486+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 shutdown_connections 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.486+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff19406be30 0x7ff19406e2f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.486+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff1a814b990 0x7ff1a8147180 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.486+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff1a806c3e0 0x7ff1a8146c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.486+0000 7ff1ae509640 1 --2- 172.21.15.160:0/2769952267 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff1a8066f90 0x7ff1a80aa8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.486+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 >> 172.21.15.160:0/2769952267 conn(0x7ff1a80c3e30 msgr2=0x7ff1a806df20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.487+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 shutdown_connections 2024-05-09T07:13:45.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.487+0000 7ff1ae509640 1 -- 172.21.15.160:0/2769952267 wait complete. 2024-05-09T07:13:45.504 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-09T07:13:45.504 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-09T07:13:45.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.650+0000 7f6957a1c640 1 Processor -- start 2024-05-09T07:13:45.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.650+0000 7f6957a1c640 1 -- start start 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f6957a1c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6950152a80 0x7f69501568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f6957a1c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6950155e70 0x7f6950159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f6957a1c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69500c7ae0 con 0x7f69500c5690 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f6957a1c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f6950152e60 con 0x7f6950152a80 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f6957a1c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f69500c80a0 con 0x7f6950155e70 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f69557b8640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6950152a80 0x7f69501568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f69557b8640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6950152a80 0x7f69501568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40090/0 (socket says 172.21.15.160:40090) 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f69557b8640 1 -- 172.21.15.160:0/568689219 learned_addr learned my addr 172.21.15.160:0/568689219 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.651+0000 7f6955fb9640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f69500c5690 0x7f695006c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35280/0 (socket says 172.21.15.160:35280) 2024-05-09T07:13:45.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f6954fb7640 1 --2- 172.21.15.160:0/568689219 >> v2:172.21.15.188:3300/0 conn(0x7f6950155e70 0x7f6950159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f69557b8640 1 -- 172.21.15.160:0/568689219 >> v2:172.21.15.188:3300/0 conn(0x7f6950155e70 msgr2=0x7f6950159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f69557b8640 1 --2- 172.21.15.160:0/568689219 >> v2:172.21.15.188:3300/0 conn(0x7f6950155e70 0x7f6950159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f69557b8640 1 -- 172.21.15.160:0/568689219 >> v1:172.21.15.160:6789/0 conn(0x7f69500c5690 legacy=0x7f695006c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f69557b8640 1 -- 172.21.15.160:0/568689219 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69500c7dc0 con 0x7f6950152a80 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f69557b8640 1 --2- 172.21.15.160:0/568689219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69501568e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69500ad340 tx=0x7f694002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13751ae08cc6944 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f694ffff640 1 -- 172.21.15.160:0/568689219 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940036070 con 0x7f6950152a80 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f694ffff640 1 -- 172.21.15.160:0/568689219 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6940031600 con 0x7f6950152a80 2024-05-09T07:13:45.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.652+0000 7f694ffff640 1 -- 172.21.15.160:0/568689219 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400319d0 con 0x7f6950152a80 2024-05-09T07:13:45.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 -- 172.21.15.160:0/568689219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 msgr2=0x7f69501568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:45.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 --2- 172.21.15.160:0/568689219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69501568e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69500ad340 tx=0x7f694002a610 comp rx=0 tx=0).stop 2024-05-09T07:13:45.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 -- 172.21.15.160:0/568689219 shutdown_connections 2024-05-09T07:13:45.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 --2- 172.21.15.160:0/568689219 >> v2:172.21.15.188:3300/0 conn(0x7f6950155e70 0x7f6950159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 --2- 172.21.15.160:0/568689219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69501568e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 -- 172.21.15.160:0/568689219 >> 172.21.15.160:0/568689219 conn(0x7f6950072ec0 msgr2=0x7f69500ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 -- 172.21.15.160:0/568689219 shutdown_connections 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.653+0000 7f6957a1c640 1 -- 172.21.15.160:0/568689219 wait complete. 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.654+0000 7f6957a1c640 1 Processor -- start 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.654+0000 7f6957a1c640 1 -- start start 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6957a1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69500c5690 0x7f69500c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6957a1c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69500abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6957a1c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6950155e70 0x7f69501ee390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6957a1c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6950156320 con 0x7f6950152a80 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6957a1c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6950158df0 con 0x7f6950155e70 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6957a1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69500ad6a0 con 0x7f69500c5690 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69500abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:45.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69500abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40098/0 (socket says 172.21.15.160:40098) 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 -- 172.21.15.160:0/3711855970 learned_addr learned my addr 172.21.15.160:0/3711855970 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6955fb9640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6950155e70 0x7f69501ee390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f69557b8640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69500c5690 0x7f69500c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 -- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69500c5690 msgr2=0x7f69500c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69500c5690 0x7f69500c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 -- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6950155e70 msgr2=0x7f69501ee390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6950155e70 0x7f69501ee390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 -- 172.21.15.160:0/3711855970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69500c8c20 con 0x7f6950152a80 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6955fb9640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6950155e70 0x7f69501ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f69557b8640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69500c5690 0x7f69500c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f6954fb7640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69500abf40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f694400d800 tx=0x7f694400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.655+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694400b6c0 con 0x7f6950152a80 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.656+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6944019070 con 0x7f6950152a80 2024-05-09T07:13:45.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.656+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694400ba80 con 0x7f6950152a80 2024-05-09T07:13:45.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.656+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69500c48b0 con 0x7f6950152a80 2024-05-09T07:13:45.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.656+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6950146060 con 0x7f6950152a80 2024-05-09T07:13:45.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.657+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6950075650 con 0x7f6950152a80 2024-05-09T07:13:45.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.657+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f694400bc20 con 0x7f6950152a80 2024-05-09T07:13:45.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.657+0000 7f694dffb640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f693c06be30 0x7f693c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:45.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.658+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5709+0+0 (secure 0 0 0) 0x7f694408b2a0 con 0x7f6950152a80 2024-05-09T07:13:45.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.658+0000 7f69557b8640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f693c06be30 0x7f693c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:45.662 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.658+0000 7f69557b8640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f693c06be30 0x7f693c06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6940033000 tx=0x7f6940009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:45.663 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.662+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6944059ce0 con 0x7f6950152a80 2024-05-09T07:13:45.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:45.845+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f69500c8df0 con 0x7f6950152a80 2024-05-09T07:13:46.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.462+0000 7f694dffb640 1 -- 172.21.15.160:0/3711855970 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) ==== 157+0+0 (secure 0 0 0) 0x7f69440595e0 con 0x7f6950152a80 2024-05-09T07:13:46.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f693c06be30 msgr2=0x7f693c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:46.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f693c06be30 0x7f693c06e2f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6940033000 tx=0x7f6940009000 comp rx=0 tx=0).stop 2024-05-09T07:13:46.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 msgr2=0x7f69500abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:46.467 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69500abf40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f694400d800 tx=0x7f694400dcc0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 shutdown_connections 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f693c06be30 0x7f693c06e2f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6950155e70 0x7f69501ee390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6950152a80 0x7f69500abf40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 --2- 172.21.15.160:0/3711855970 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69500c5690 0x7f69500c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.465+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 >> 172.21.15.160:0/3711855970 conn(0x7f6950072ec0 msgr2=0x7f695014b920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.466+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 shutdown_connections 2024-05-09T07:13:46.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.466+0000 7f6957a1c640 1 -- 172.21.15.160:0/3711855970 wait complete. 2024-05-09T07:13:46.480 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T07:13:46.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 Processor -- start 2024-05-09T07:13:46.591 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 -- start start 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c98157390 0x7f8c980c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c980c4090 0x7f8c980c8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c9806ffe0 con 0x7f8c980a82c0 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8c980ab760 con 0x7f8c98157390 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.590+0000 7f8c9ea07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9806f460 con 0x7f8c980c4090 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c97fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c98157390 0x7f8c980c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c9cfa4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c980c4090 0x7f8c980c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c8ffff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8c980a82c0 0x7f8c980c7900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:42608/0 (socket says 172.21.15.160:42608) 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c97fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8c98157390 0x7f8c980c3b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:55260/0 (socket says 172.21.15.160:55260) 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c9cfa4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c980c4090 0x7f8c980c8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:59340/0 (socket says 172.21.15.160:59340) 2024-05-09T07:13:46.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c8ffff640 1 -- 172.21.15.160:0/1120079590 learned_addr learned my addr 172.21.15.160:0/1120079590 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:46.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c97fff640 1 -- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c980c4090 msgr2=0x7f8c980c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:46.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c97fff640 1 --2- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c980c4090 0x7f8c980c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c97fff640 1 -- 172.21.15.160:0/1120079590 >> v1:172.21.15.188:6789/0 conn(0x7f8c980a82c0 legacy=0x7f8c980c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:46.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.591+0000 7f8c97fff640 1 -- 172.21.15.160:0/1120079590 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c9806f740 con 0x7f8c98157390 2024-05-09T07:13:46.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c97fff640 1 --2- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c98157390 0x7f8c980c3b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c880097a0 tx=0x7f8c8802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64f04b50242b31a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:46.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c977fe640 1 -- 172.21.15.160:0/1120079590 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88037070 con 0x7f8c98157390 2024-05-09T07:13:46.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c977fe640 1 -- 172.21.15.160:0/1120079590 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c88032440 con 0x7f8c98157390 2024-05-09T07:13:46.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c977fe640 1 -- 172.21.15.160:0/1120079590 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88032810 con 0x7f8c98157390 2024-05-09T07:13:46.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c98157390 msgr2=0x7f8c980c3b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c98157390 0x7f8c980c3b50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8c880097a0 tx=0x7f8c8802a180 comp rx=0 tx=0).stop 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1120079590 shutdown_connections 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c980c4090 0x7f8c980c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1120079590 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c98157390 0x7f8c980c3b50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.592+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1120079590 >> 172.21.15.160:0/1120079590 conn(0x7f8c980aa810 msgr2=0x7f8c980aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.593+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1120079590 shutdown_connections 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.593+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1120079590 wait complete. 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 Processor -- start 2024-05-09T07:13:46.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 -- start start 2024-05-09T07:13:46.598 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980a82c0 0x7f8c980c1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 0x7f8c980beee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98157390 0x7f8c98150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8c980c8550 con 0x7f8c980c4090 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9806fd00 con 0x7f8c980a82c0 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9ea07640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9806f0f0 con 0x7f8c98157390 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 0x7f8c980beee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 0x7f8c980beee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40100/0 (socket says 172.21.15.160:40100) 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c97fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980a82c0 0x7f8c980c1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c8ffff640 1 -- 172.21.15.160:0/1394686999 learned_addr learned my addr 172.21.15.160:0/1394686999 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.594+0000 7f8c9cfa4640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98157390 0x7f8c98150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c8ffff640 1 -- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98157390 msgr2=0x7f8c98150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c8ffff640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98157390 0x7f8c98150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c8ffff640 1 -- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980a82c0 msgr2=0x7f8c980c1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:46.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c8ffff640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980a82c0 0x7f8c980c1270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c8ffff640 1 -- 172.21.15.160:0/1394686999 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c980c17b0 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c97fff640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980a82c0 0x7f8c980c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c8ffff640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 0x7f8c980beee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c470 tx=0x7f8c8000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80017070 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c80005ce0 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c980c0110 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c800072d0 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.595+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c980be6e0 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.596+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c980be4f0 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.596+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c8001a070 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.597+0000 7f8c957fa640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c7806be60 0x7f8c7806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.597+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 6007+0+0 (secure 0 0 0) 0x7f8c8008a180 con 0x7f8c980c4090 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.597+0000 7f8c97fff640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c7806be60 0x7f8c7806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:46.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.597+0000 7f8c97fff640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c7806be60 0x7f8c7806e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8c88034000 tx=0x7f8c8802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:46.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.601+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8c800583b0 con 0x7f8c980c4090 2024-05-09T07:13:46.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:46.774+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8c980c6ca0 con 0x7f8c980c4090 2024-05-09T07:13:48.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.479+0000 7f8c957fa640 1 -- 172.21.15.160:0/1394686999 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) ==== 228+0+0 (secure 0 0 0) 0x7f8c8005c6b0 con 0x7f8c980c4090 2024-05-09T07:13:48.481 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_data_ec' created 2024-05-09T07:13:48.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c7806be60 msgr2=0x7f8c7806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:48.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c7806be60 0x7f8c7806e320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8c88034000 tx=0x7f8c8802e000 comp rx=0 tx=0).stop 2024-05-09T07:13:48.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 msgr2=0x7f8c980beee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:48.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 0x7f8c980beee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c470 tx=0x7f8c8000c930 comp rx=0 tx=0).stop 2024-05-09T07:13:48.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 shutdown_connections 2024-05-09T07:13:48.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c7806be60 0x7f8c7806e320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:48.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c98157390 0x7f8c98150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:48.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c980c4090 0x7f8c980beee0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000c470 tx=0x7f8c8000c930 comp rx=0 tx=0).stop 2024-05-09T07:13:48.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 --2- 172.21.15.160:0/1394686999 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c980a82c0 0x7f8c980c1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:48.487 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.484+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 >> 172.21.15.160:0/1394686999 conn(0x7f8c980aa810 msgr2=0x7f8c980aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:48.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.489+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 shutdown_connections 2024-05-09T07:13:48.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.489+0000 7f8c9ea07640 1 -- 172.21.15.160:0/1394686999 wait complete. 2024-05-09T07:13:48.505 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-09T07:13:48.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.611+0000 7f2bc9012640 1 Processor -- start 2024-05-09T07:13:48.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc9012640 1 -- start start 2024-05-09T07:13:48.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc9012640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2bc40719e0 0x7f2bc40c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:48.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc9012640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc41595e0 con 0x7f2bc406b160 2024-05-09T07:13:48.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc9012640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bc41598c0 con 0x7f2bc4071f80 2024-05-09T07:13:48.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc9012640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f2bc40730a0 con 0x7f2bc40719e0 2024-05-09T07:13:48.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc359e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f2bc40719e0 0x7f2bc40c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc2d9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f2bc406b160 0x7f2bc4155540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35292/0 (socket says 172.21.15.160:35292) 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.612+0000 7f2bc2d9d640 1 -- 172.21.15.160:0/3577369849 learned_addr learned my addr 172.21.15.160:0/3577369849 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.613+0000 7f2bc359e640 1 -- 172.21.15.160:0/3577369849 >> v1:172.21.15.188:6789/0 conn(0x7f2bc4071f80 legacy=0x7f2bc415deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.613+0000 7f2bc359e640 1 -- 172.21.15.160:0/3577369849 >> v1:172.21.15.160:6789/0 conn(0x7f2bc406b160 legacy=0x7f2bc4155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.613+0000 7f2bc359e640 1 -- 172.21.15.160:0/3577369849 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc40732f0 con 0x7f2bc40719e0 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.613+0000 7f2bc1d9b640 1 -- 172.21.15.160:0/3577369849 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1570757589 0 0) 0x7f2bc41595e0 con 0x7f2bc406b160 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.613+0000 7f2bc359e640 1 --2- 172.21.15.160:0/3577369849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc40719e0 0x7f2bc40c6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009d90 tx=0x7f2bbc02a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a5bca5f9717fc71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc1d9b640 1 -- 172.21.15.160:0/3577369849 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc007740 con 0x7f2bc40719e0 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc1d9b640 1 -- 172.21.15.160:0/3577369849 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc0078e0 con 0x7f2bc40719e0 2024-05-09T07:13:48.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc1d9b640 1 -- 172.21.15.160:0/3577369849 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc007cb0 con 0x7f2bc40719e0 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 -- 172.21.15.160:0/3577369849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc40719e0 msgr2=0x7f2bc40c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 --2- 172.21.15.160:0/3577369849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc40719e0 0x7f2bc40c6180 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009d90 tx=0x7f2bbc02a580 comp rx=0 tx=0).stop 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 -- 172.21.15.160:0/3577369849 shutdown_connections 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 --2- 172.21.15.160:0/3577369849 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc40719e0 0x7f2bc40c6180 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 -- 172.21.15.160:0/3577369849 >> 172.21.15.160:0/3577369849 conn(0x7f2bc40a98a0 msgr2=0x7f2bc40a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 -- 172.21.15.160:0/3577369849 shutdown_connections 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.614+0000 7f2bc9012640 1 -- 172.21.15.160:0/3577369849 wait complete. 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.615+0000 7f2bc9012640 1 Processor -- start 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 -- start start 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc406b160 0x7f2bc41572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:48.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc40719e0 0x7f2bc414d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 0x7f2bc40abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4152e30 con 0x7f2bc4071f80 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4073660 con 0x7f2bc40719e0 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc9012640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2bc4152fb0 con 0x7f2bc406b160 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 0x7f2bc40abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 0x7f2bc40abbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40112/0 (socket says 172.21.15.160:40112) 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.616+0000 7f2bc359e640 1 -- 172.21.15.160:0/1736002474 learned_addr learned my addr 172.21.15.160:0/1736002474 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc259c640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc40719e0 0x7f2bc414d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc2d9d640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc406b160 0x7f2bc41572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc359e640 1 -- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc406b160 msgr2=0x7f2bc41572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc359e640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc406b160 0x7f2bc41572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc359e640 1 -- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc40719e0 msgr2=0x7f2bc414d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:48.621 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc359e640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc40719e0 0x7f2bc414d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc359e640 1 -- 172.21.15.160:0/1736002474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc4158220 con 0x7f2bc4071f80 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc259c640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc40719e0 0x7f2bc414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc359e640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 0x7f2bc40abbf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc02ab70 tx=0x7f2bbc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc040070 con 0x7f2bc4071f80 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2bbc031c90 con 0x7f2bc4071f80 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc045680 con 0x7f2bc4071f80 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc4158500 con 0x7f2bc4071f80 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.617+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc40ab6d0 con 0x7f2bc4071f80 2024-05-09T07:13:48.622 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.618+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc4075650 con 0x7f2bc4071f80 2024-05-09T07:13:48.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.619+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2bbc039040 con 0x7f2bc4071f80 2024-05-09T07:13:48.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.619+0000 7f2bbb7fe640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2ba406be30 0x7f2ba406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:48.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.619+0000 7f2bc2d9d640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2ba406be30 0x7f2ba406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:48.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.619+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 6500+0+0 (secure 0 0 0) 0x7f2bbc0acbd0 con 0x7f2bc4071f80 2024-05-09T07:13:48.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.620+0000 7f2bc2d9d640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2ba406be30 0x7f2ba406e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2bac000f60 tx=0x7f2bac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:48.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.624+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2bbc07b320 con 0x7f2bc4071f80 2024-05-09T07:13:48.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:48.800+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f2bc4072e80 con 0x7f2bc4071f80 2024-05-09T07:13:49.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.484+0000 7f2bbb7fe640 1 -- 172.21.15.160:0/1736002474 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) ==== 169+0+0 (secure 0 0 0) 0x7f2bbc07ac20 con 0x7f2bc4071f80 2024-05-09T07:13:49.486 INFO:teuthology.orchestra.run.smithi160.stderr:set pool 4 allow_ec_overwrites to true 2024-05-09T07:13:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2ba406be30 msgr2=0x7f2ba406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2ba406be30 0x7f2ba406e2f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2bac000f60 tx=0x7f2bac009040 comp rx=0 tx=0).stop 2024-05-09T07:13:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 msgr2=0x7f2bc40abbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 0x7f2bc40abbf0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc02ab70 tx=0x7f2bbc004a40 comp rx=0 tx=0).stop 2024-05-09T07:13:49.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 shutdown_connections 2024-05-09T07:13:49.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f2ba406be30 0x7f2ba406e2f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2bc40719e0 0x7f2bc414d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2bc4071f80 0x7f2bc40abbf0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 --2- 172.21.15.160:0/1736002474 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2bc406b160 0x7f2bc41572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.493+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 >> 172.21.15.160:0/1736002474 conn(0x7f2bc40a98a0 msgr2=0x7f2bc406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:49.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.496+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 shutdown_connections 2024-05-09T07:13:49.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.496+0000 7f2bc9012640 1 -- 172.21.15.160:0/1736002474 wait complete. 2024-05-09T07:13:49.519 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-09T07:13:49.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.621+0000 7fc4bd7e3640 1 Processor -- start 2024-05-09T07:13:49.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.621+0000 7fc4bd7e3640 1 -- start start 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4bd7e3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc4b806c3e0 0x7fc4b80c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4bd7e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b8066f90 0x7fc4b80c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4bd7e3640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc4b814b990 0x7fc4b814bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4bd7e3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b80ac3b0 con 0x7fc4b806c3e0 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4bd7e3640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc4b8155f10 con 0x7fc4b814b990 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4bd7e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b80ac650 con 0x7fc4b8066f90 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc4b806c3e0 0x7fc4b80c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc4b806c3e0 0x7fc4b80c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40118/0 (socket says 172.21.15.160:40118) 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/3142556210 learned_addr learned my addr 172.21.15.160:0/3142556210 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/3142556210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b8066f90 msgr2=0x7fc4b80c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 --2- 172.21.15.160:0/3142556210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b8066f90 0x7fc4b80c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/3142556210 >> v2:172.21.15.188:3300/0 conn(0x7fc4b814b990 msgr2=0x7fc4b814bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:49.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 --2- 172.21.15.160:0/3142556210 >> v2:172.21.15.188:3300/0 conn(0x7fc4b814b990 0x7fc4b814bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.622+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/3142556210 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b81561f0 con 0x7fc4b806c3e0 2024-05-09T07:13:49.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.623+0000 7fc4b6ffd640 1 --2- 172.21.15.160:0/3142556210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b806c3e0 0x7fc4b80c20f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4b806cc30 tx=0x7fc4a802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e2bffcd3ca5a8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:49.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.623+0000 7fc4b67fc640 1 -- 172.21.15.160:0/3142556210 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8037070 con 0x7fc4b806c3e0 2024-05-09T07:13:49.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.623+0000 7fc4b67fc640 1 -- 172.21.15.160:0/3142556210 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a8032440 con 0x7fc4b806c3e0 2024-05-09T07:13:49.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.623+0000 7fc4b67fc640 1 -- 172.21.15.160:0/3142556210 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8032810 con 0x7fc4b806c3e0 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/3142556210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b806c3e0 msgr2=0x7fc4b80c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/3142556210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b806c3e0 0x7fc4b80c20f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc4b806cc30 tx=0x7fc4a802a140 comp rx=0 tx=0).stop 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/3142556210 shutdown_connections 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/3142556210 >> v2:172.21.15.188:3300/0 conn(0x7fc4b814b990 0x7fc4b814bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/3142556210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b8066f90 0x7fc4b80c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/3142556210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b806c3e0 0x7fc4b80c20f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/3142556210 >> 172.21.15.160:0/3142556210 conn(0x7fc4b80c3e30 msgr2=0x7fc4b80c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/3142556210 shutdown_connections 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.624+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/3142556210 wait complete. 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.625+0000 7fc4bd7e3640 1 Processor -- start 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.625+0000 7fc4bd7e3640 1 -- start start 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 0x7fc4b813ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b806c3e0 0x7fc4b81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b814b990 0x7fc4b81ee480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80c4d80 con 0x7fc4b8066f90 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc4b80c4c00 con 0x7fc4b806c3e0 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc4b806c7c0 con 0x7fc4b814b990 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 0x7fc4b813ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 0x7fc4b813ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40120/0 (socket says 172.21.15.160:40120) 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/2135839976 learned_addr learned my addr 172.21.15.160:0/2135839976 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:49.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4ae7fc640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b806c3e0 0x7fc4b81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b77fe640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b814b990 0x7fc4b81ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b814b990 msgr2=0x7fc4b81ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b814b990 0x7fc4b81ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b806c3e0 msgr2=0x7fc4b81531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b806c3e0 0x7fc4b81531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 -- 172.21.15.160:0/2135839976 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4b81eaf10 con 0x7fc4b8066f90 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4ae7fc640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b806c3e0 0x7fc4b81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b77fe640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b814b990 0x7fc4b81ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4b6ffd640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 0x7fc4b813ff00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8000c00 tx=0x7fc4a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8041070 con 0x7fc4b8066f90 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4a803c870 con 0x7fc4b8066f90 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4b81571f0 con 0x7fc4b8066f90 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.626+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8037040 con 0x7fc4b8066f90 2024-05-09T07:13:49.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.627+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4b81577b0 con 0x7fc4b8066f90 2024-05-09T07:13:49.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.627+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4b8075650 con 0x7fc4b8066f90 2024-05-09T07:13:49.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.628+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc4a8044070 con 0x7fc4b8066f90 2024-05-09T07:13:49.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.628+0000 7fc4affff640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc49806be30 0x7fc49806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:49.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.628+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7fc4a80abd80 con 0x7fc4b8066f90 2024-05-09T07:13:49.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.628+0000 7fc4ae7fc640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc49806be30 0x7fc49806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:49.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.629+0000 7fc4ae7fc640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc49806be30 0x7fc49806e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:49.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.633+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc4a807a4a0 con 0x7fc4b8066f90 2024-05-09T07:13:49.817 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-05-09T07:13:49.815+0000 7fa6bee05640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T07:13:49.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:49.845+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fc4b806fb20 con 0x7fc4b8066f90 2024-05-09T07:13:50.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.493+0000 7fc4affff640 1 -- 172.21.15.160:0/2135839976 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v6) ==== 137+0+0 (secure 0 0 0) 0x7fc4a8079da0 con 0x7fc4b8066f90 2024-05-09T07:13:50.495 INFO:teuthology.orchestra.run.smithi160.stderr:added data pool 4 to fsmap 2024-05-09T07:13:50.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc49806be30 msgr2=0x7fc49806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc49806be30 0x7fc49806e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0008040 comp rx=0 tx=0).stop 2024-05-09T07:13:50.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 msgr2=0x7fc4b813ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 0x7fc4b813ff00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8000c00 tx=0x7fc4a80046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 shutdown_connections 2024-05-09T07:13:50.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc49806be30 0x7fc49806e2f0 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0006fd0 tx=0x7fc4a0008040 comp rx=0 tx=0).stop 2024-05-09T07:13:50.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc4b806c3e0 0x7fc4b81531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc4b814b990 0x7fc4b81ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 --2- 172.21.15.160:0/2135839976 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4b8066f90 0x7fc4b813ff00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 >> 172.21.15.160:0/2135839976 conn(0x7fc4b80c3e30 msgr2=0x7fc4b8149f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:50.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 shutdown_connections 2024-05-09T07:13:50.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.495+0000 7fc4bd7e3640 1 -- 172.21.15.160:0/2135839976 wait complete. 2024-05-09T07:13:50.510 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:13:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.612+0000 7fce0a714640 1 Processor -- start 2024-05-09T07:13:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.612+0000 7fce0a714640 1 -- start start 2024-05-09T07:13:50.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce0a714640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce0a714640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fce0406b1f0 0x7fce0406e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce0a714640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fce04155350 0x7fce04159140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce0a714640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fce04155bf0 con 0x7fce0406b1f0 2024-05-09T07:13:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce0a714640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fce04155ed0 con 0x7fce04155350 2024-05-09T07:13:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce0a714640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce041561b0 con 0x7fce0406e400 2024-05-09T07:13:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fce0406b1f0 0x7fce0406e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fce0406b1f0 0x7fce0406e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40126/0 (socket says 172.21.15.160:40126) 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 -- 172.21.15.160:0/4265929996 learned_addr learned my addr 172.21.15.160:0/4265929996 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce03fff640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce08cb1640 1 --2- 172.21.15.160:0/4265929996 >> v2:172.21.15.188:3300/0 conn(0x7fce04155350 0x7fce04159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 -- 172.21.15.160:0/4265929996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 msgr2=0x7fce040c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 -- 172.21.15.160:0/4265929996 >> v2:172.21.15.188:3300/0 conn(0x7fce04155350 msgr2=0x7fce04159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 --2- 172.21.15.160:0/4265929996 >> v2:172.21.15.188:3300/0 conn(0x7fce04155350 0x7fce04159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fcdfbfff640 1 -- 172.21.15.160:0/4265929996 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce04158940 con 0x7fce0406b1f0 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.613+0000 7fce03fff640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.614+0000 7fcdfbfff640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce0406b1f0 0x7fce0406e7e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce0406df90 tx=0x7fcdec02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641e85a70f38c637 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.614+0000 7fce037fe640 1 -- 172.21.15.160:0/4265929996 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec037070 con 0x7fce0406b1f0 2024-05-09T07:13:50.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.614+0000 7fce037fe640 1 -- 172.21.15.160:0/4265929996 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdec032440 con 0x7fce0406b1f0 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.614+0000 7fce037fe640 1 -- 172.21.15.160:0/4265929996 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec032810 con 0x7fce0406b1f0 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 -- 172.21.15.160:0/4265929996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce0406b1f0 msgr2=0x7fce0406e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce0406b1f0 0x7fce0406e7e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fce0406df90 tx=0x7fcdec02a140 comp rx=0 tx=0).stop 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 -- 172.21.15.160:0/4265929996 shutdown_connections 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 --2- 172.21.15.160:0/4265929996 >> v2:172.21.15.188:3300/0 conn(0x7fce04155350 0x7fce04159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce0406b1f0 0x7fce0406e7e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 --2- 172.21.15.160:0/4265929996 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 -- 172.21.15.160:0/4265929996 >> 172.21.15.160:0/4265929996 conn(0x7fce040c76e0 msgr2=0x7fce040c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 -- 172.21.15.160:0/4265929996 shutdown_connections 2024-05-09T07:13:50.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.615+0000 7fce0a714640 1 -- 172.21.15.160:0/4265929996 wait complete. 2024-05-09T07:13:50.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.616+0000 7fce0a714640 1 Processor -- start 2024-05-09T07:13:50.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.616+0000 7fce0a714640 1 -- start start 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0406b1f0 0x7fce041eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 0x7fce040c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fce04155960 con 0x7fce04155350 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fce041557e0 con 0x7fce0406b1f0 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fce040c2e60 con 0x7fce0406e400 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 0x7fce040c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 0x7fce040c5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40134/0 (socket says 172.21.15.160:40134) 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 -- 172.21.15.160:0/143055744 learned_addr learned my addr 172.21.15.160:0/143055744 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce03fff640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0406b1f0 0x7fce041eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fcdfbfff640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 -- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 msgr2=0x7fce040c6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 -- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0406b1f0 msgr2=0x7fce041eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0406b1f0 0x7fce041eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 -- 172.21.15.160:0/143055744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce041ebdf0 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fcdfbfff640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce03fff640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0406b1f0 0x7fce041eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce08cb1640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 0x7fce040c5850 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00efc0 tx=0x7fcdfc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc018020 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdfc014480 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce0414a070 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc014870 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.617+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce040c5050 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.618+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce040756c0 con 0x7fce04155350 2024-05-09T07:13:50.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.619+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcdfc005d10 con 0x7fce04155350 2024-05-09T07:13:50.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.619+0000 7fce017fa640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcde406be30 0x7fcde406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.619+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7fcdfc08bb60 con 0x7fce04155350 2024-05-09T07:13:50.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.619+0000 7fce03fff640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcde406be30 0x7fcde406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.620+0000 7fce03fff640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcde406be30 0x7fcde406e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400c940 tx=0x7fcdf4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:50.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.624+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcdfc022070 con 0x7fce04155350 2024-05-09T07:13:50.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.835+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce040675a0 con 0x7fce04155350 2024-05-09T07:13:50.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.836+0000 7fce017fa640 1 -- 172.21.15.160:0/143055744 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5183 (secure 0 0 0) 0x7fcdfc010040 con 0x7fce04155350 2024-05-09T07:13:50.838 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:50.838 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:13:49.846482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:13:50.838 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 6 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcde406be30 msgr2=0x7fcde406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcde406be30 0x7fcde406e2f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdf400c940 tx=0x7fcdf4008040 comp rx=0 tx=0).stop 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 msgr2=0x7fce040c5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 0x7fce040c5850 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00efc0 tx=0x7fcdfc00c490 comp rx=0 tx=0).stop 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 shutdown_connections 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcde406be30 0x7fcde406e2f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce04155350 0x7fce040c5850 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fce0406e400 0x7fce040c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 --2- 172.21.15.160:0/143055744 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fce0406b1f0 0x7fce041eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 >> 172.21.15.160:0/143055744 conn(0x7fce040c76e0 msgr2=0x7fce0406b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:50.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 shutdown_connections 2024-05-09T07:13:50.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.838+0000 7fce0a714640 1 -- 172.21.15.160:0/143055744 wait complete. 2024-05-09T07:13:50.851 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:50.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 Processor -- start 2024-05-09T07:13:50.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 -- start start 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f63800be320 0x7f63800c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f63800c8b90 0x7f6380155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6380159840 con 0x7f63800c5610 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f6380159310 con 0x7f63800c8b90 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f63864bd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6380159560 con 0x7f63800be320 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f637ffff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f63800c5610 0x7f6380155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35308/0 (socket says 172.21.15.160:35308) 2024-05-09T07:13:50.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.950+0000 7f637ffff640 1 -- 172.21.15.160:0/3486960311 learned_addr learned my addr 172.21.15.160:0/3486960311 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f6384a5a640 1 --2- 172.21.15.160:0/3486960311 >> v2:172.21.15.160:3300/0 conn(0x7f63800c8b90 0x7f6380155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f6377fff640 1 --2- 172.21.15.160:0/3486960311 >> v2:172.21.15.188:3300/0 conn(0x7f63800be320 0x7f63800c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f637f7fe640 1 -- 172.21.15.160:0/3486960311 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1266575434 0 0) 0x7f6380159840 con 0x7f63800c5610 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f637f7fe640 1 -- 172.21.15.160:0/3486960311 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f636c003520 con 0x7f63800c5610 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f6384a5a640 1 -- 172.21.15.160:0/3486960311 >> v2:172.21.15.188:3300/0 conn(0x7f63800be320 msgr2=0x7f63800c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f6384a5a640 1 --2- 172.21.15.160:0/3486960311 >> v2:172.21.15.188:3300/0 conn(0x7f63800be320 0x7f63800c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f6384a5a640 1 -- 172.21.15.160:0/3486960311 >> v1:172.21.15.160:6789/0 conn(0x7f63800c5610 legacy=0x7f6380155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.951+0000 7f6384a5a640 1 -- 172.21.15.160:0/3486960311 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63800c77d0 con 0x7f63800c8b90 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f6384a5a640 1 --2- 172.21.15.160:0/3486960311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800c8b90 0x7f6380155a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f638006d860 tx=0x7f637802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bee90f84761c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f637f7fe640 1 -- 172.21.15.160:0/3486960311 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378007740 con 0x7f63800c8b90 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f637f7fe640 1 -- 172.21.15.160:0/3486960311 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63780078e0 con 0x7f63800c8b90 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f637f7fe640 1 -- 172.21.15.160:0/3486960311 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6378007cb0 con 0x7f63800c8b90 2024-05-09T07:13:50.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 -- 172.21.15.160:0/3486960311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800c8b90 msgr2=0x7f6380155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 --2- 172.21.15.160:0/3486960311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800c8b90 0x7f6380155a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f638006d860 tx=0x7f637802a560 comp rx=0 tx=0).stop 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 -- 172.21.15.160:0/3486960311 shutdown_connections 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 --2- 172.21.15.160:0/3486960311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800c8b90 0x7f6380155a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 --2- 172.21.15.160:0/3486960311 >> v2:172.21.15.188:3300/0 conn(0x7f63800be320 0x7f63800c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 -- 172.21.15.160:0/3486960311 >> 172.21.15.160:0/3486960311 conn(0x7f6380072e90 msgr2=0x7f63800ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 -- 172.21.15.160:0/3486960311 shutdown_connections 2024-05-09T07:13:50.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.952+0000 7f63864bd640 1 -- 172.21.15.160:0/3486960311 wait complete. 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.953+0000 7f63864bd640 1 Processor -- start 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.953+0000 7f63864bd640 1 -- start start 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f63864bd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 0x7f63801eb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f63864bd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63800c5610 0x7f63801579a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f63864bd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800c8b90 0x7f6380157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f63864bd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f63800ad6a0 con 0x7f63800be320 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f63864bd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f63800c7d90 con 0x7f63800c5610 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f63864bd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6380155f60 con 0x7f63800c8b90 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 0x7f63801eb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 0x7f63801eb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40146/0 (socket says 172.21.15.160:40146) 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 -- 172.21.15.160:0/1542741608 learned_addr learned my addr 172.21.15.160:0/1542741608 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:50.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f6377fff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63800c5610 0x7f63801579a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f6384a5a640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800c8b90 0x7f6380157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 -- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800c8b90 msgr2=0x7f6380157ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800c8b90 0x7f6380157ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 -- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63800c5610 msgr2=0x7f63801579a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.954+0000 7f637ffff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63800c5610 0x7f63801579a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f637ffff640 1 -- 172.21.15.160:0/1542741608 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63800cab50 con 0x7f63800be320 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f6377fff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63800c5610 0x7f63801579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f637ffff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 0x7f63801eb0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f637000c460 tx=0x7f637000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370017070 con 0x7f63800be320 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6370005ce0 con 0x7f63800be320 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6370007290 con 0x7f63800be320 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6380153220 con 0x7f63800be320 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.955+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6380154f90 con 0x7f63800be320 2024-05-09T07:13:50.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.956+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63800756c0 con 0x7f63800be320 2024-05-09T07:13:50.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.956+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f637001a070 con 0x7f63800be320 2024-05-09T07:13:50.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.957+0000 7f637d7fa640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:50.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.957+0000 7f6377fff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:50.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.957+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f637008a470 con 0x7f63800be320 2024-05-09T07:13:50.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.957+0000 7f6377fff640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f636006be30 0x7f636006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f638014ea00 tx=0x7f6368008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:50.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:50.961+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6370058b70 con 0x7f63800be320 2024-05-09T07:13:51.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.136+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6380074b90 con 0x7f63800be320 2024-05-09T07:13:51.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.137+0000 7f637d7fa640 1 -- 172.21.15.160:0/1542741608 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18563 (secure 0 0 0) 0x7f6370058470 con 0x7f63800be320 2024-05-09T07:13:51.139 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:51.139 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":24,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:49.846283+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:51.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f636006be30 msgr2=0x7f636006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f636006be30 0x7f636006e2f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f638014ea00 tx=0x7f6368008040 comp rx=0 tx=0).stop 2024-05-09T07:13:51.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 msgr2=0x7f63801eb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 0x7f63801eb0c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f637000c460 tx=0x7f637000c920 comp rx=0 tx=0).stop 2024-05-09T07:13:51.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 shutdown_connections 2024-05-09T07:13:51.141 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f636006be30 0x7f636006e2f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f63800c5610 0x7f63801579a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f63800c8b90 0x7f6380157ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 --2- 172.21.15.160:0/1542741608 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f63800be320 0x7f63801eb0c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 >> 172.21.15.160:0/1542741608 conn(0x7f6380072e90 msgr2=0x7f638014efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:51.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 shutdown_connections 2024-05-09T07:13:51.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.139+0000 7f63864bd640 1 -- 172.21.15.160:0/1542741608 wait complete. 2024-05-09T07:13:51.152 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:51.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.252+0000 7f277e64d640 1 Processor -- start 2024-05-09T07:13:51.254 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277e64d640 1 -- start start 2024-05-09T07:13:51.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277e64d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 0x7f27780c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277e64d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f27780be320 0x7f2778155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277e64d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2778159840 con 0x7f27780c8b90 2024-05-09T07:13:51.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277e64d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2778159310 con 0x7f27780be320 2024-05-09T07:13:51.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277e64d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2778159560 con 0x7f27780c5610 2024-05-09T07:13:51.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277cbea640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f27780c8b90 0x7f2778155850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35312/0 (socket says 172.21.15.160:35312) 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.253+0000 7f277cbea640 1 -- 172.21.15.160:0/740845732 learned_addr learned my addr 172.21.15.160:0/740845732 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 --2- 172.21.15.160:0/740845732 >> v2:172.21.15.188:3300/0 conn(0x7f27780be320 0x7f2778155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f2777fff640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 0x7f27780c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 118246281 0 0) 0x7f2778159840 con 0x7f27780c8b90 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2764003520 con 0x7f27780c8b90 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2987295560 0 0) 0x7f2764003520 con 0x7f27780c8b90 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 msgr2=0x7f27780c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 0x7f27780c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 >> v2:172.21.15.188:3300/0 conn(0x7f27780be320 msgr2=0x7f2778155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 --2- 172.21.15.160:0/740845732 >> v2:172.21.15.188:3300/0 conn(0x7f27780be320 0x7f2778155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27780c77d0 con 0x7f27780c8b90 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f27700019f0 con 0x7f27780c8b90 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 --2- 172.21.15.160:0/740845732 >> v2:172.21.15.188:3300/0 conn(0x7f27780be320 0x7f2778155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:51.259 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 >> v1:172.21.15.160:6789/0 conn(0x7f27780c8b90 legacy=0x7f2778155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 0x7f27640039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2764003f30 0x7f2764004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764004d40 0x7f27640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f2778159310 con 0x7f2764003f30 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2778159560 con 0x7f2764004d40 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f276400baa0 con 0x7f27780c5610 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2764003f30 0x7f2764004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f277cbea640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764004d40 0x7f27640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f2777fff640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 0x7f27640039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 -- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 msgr2=0x7f27640039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780c5610 0x7f27640039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 -- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764004d40 msgr2=0x7f27640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.260 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764004d40 0x7f27640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.254+0000 7f276ffff640 1 -- 172.21.15.160:0/740845732 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2764003520 con 0x7f2764003f30 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.255+0000 7f276ffff640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2764003f30 0x7f2764004800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2760009c60 tx=0x7f2764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d051a2bffe08153 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.255+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760036070 con 0x7f2764003f30 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.255+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2760031400 con 0x7f2764003f30 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.255+0000 7f27777fe640 1 -- 172.21.15.160:0/740845732 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2760009860 con 0x7f2764003f30 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277e64d640 1 -- 172.21.15.160:0/740845732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2764003f30 msgr2=0x7f2764004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277e64d640 1 --2- 172.21.15.160:0/740845732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2764003f30 0x7f2764004800 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2760009c60 tx=0x7f2764005830 comp rx=0 tx=0).stop 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277cbea640 1 -- 172.21.15.160:0/740845732 reap_dead start 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277e64d640 1 -- 172.21.15.160:0/740845732 shutdown_connections 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277e64d640 1 -- 172.21.15.160:0/740845732 >> 172.21.15.160:0/740845732 conn(0x7f2778072e90 msgr2=0x7f27780ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277e64d640 1 -- 172.21.15.160:0/740845732 shutdown_connections 2024-05-09T07:13:51.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.256+0000 7f277e64d640 1 -- 172.21.15.160:0/740845732 wait complete. 2024-05-09T07:13:51.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.257+0000 7f277e64d640 1 Processor -- start 2024-05-09T07:13:51.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.257+0000 7f277e64d640 1 -- start start 2024-05-09T07:13:51.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277e64d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764003f30 0x7f2778155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277e64d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780bdcc0 0x7f27781e7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277e64d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 0x7f27780bf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277e64d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f27780ad6a0 con 0x7f27781e8200 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277e64d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f27780c7d90 con 0x7f2764003f30 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277e64d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2778155f60 con 0x7f27780bdcc0 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 0x7f27780bf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 0x7f27780bf1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40158/0 (socket says 172.21.15.160:40158) 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 -- 172.21.15.160:0/3153841779 learned_addr learned my addr 172.21.15.160:0/3153841779 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f2777fff640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764003f30 0x7f2778155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f276ffff640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780bdcc0 0x7f27781e7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 -- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780bdcc0 msgr2=0x7f27781e7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.265 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780bdcc0 0x7f27781e7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 -- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764003f30 msgr2=0x7f2778155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764003f30 0x7f2778155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 -- 172.21.15.160:0/3153841779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27781e8690 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f2777fff640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764003f30 0x7f2778155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f277cbea640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 0x7f27780bf1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2770000f80 tx=0x7f2770007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277000ab00 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2770019070 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.258+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27700133f0 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.259+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2778154100 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.259+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27781546c0 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.259+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27780ad340 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.260+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2770013590 con 0x7f27781e8200 2024-05-09T07:13:51.266 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.260+0000 7f27757fa640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f275806be30 0x7f275806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.260+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6532+0+0 (secure 0 0 0) 0x7f277008b720 con 0x7f27781e8200 2024-05-09T07:13:51.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.260+0000 7f2777fff640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f275806be30 0x7f275806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.261+0000 7f2777fff640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f275806be30 0x7f275806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2768009ae0 tx=0x7f2768008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.265+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2770059d90 con 0x7f27781e8200 2024-05-09T07:13:51.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.439+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27780756a0 con 0x7f27781e8200 2024-05-09T07:13:51.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.441+0000 7f27757fa640 1 -- 172.21.15.160:0/3153841779 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) ==== 74+0+18563 (secure 0 0 0) 0x7f2770059690 con 0x7f27781e8200 2024-05-09T07:13:51.442 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:51.443 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":24,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:49.846283+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":32773,"flags_names":"hashpspool,ec_overwrites,creating","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:51.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f275806be30 msgr2=0x7f275806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f275806be30 0x7f275806e2f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2768009ae0 tx=0x7f2768008040 comp rx=0 tx=0).stop 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 msgr2=0x7f27780bf1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 0x7f27780bf1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2770000f80 tx=0x7f2770007b00 comp rx=0 tx=0).stop 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 shutdown_connections 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f275806be30 0x7f275806e2f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27781e8200 0x7f27780bf1f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f27780bdcc0 0x7f27781e7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 --2- 172.21.15.160:0/3153841779 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2764003f30 0x7f2778155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 >> 172.21.15.160:0/3153841779 conn(0x7f2778072e90 msgr2=0x7f27780aafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 shutdown_connections 2024-05-09T07:13:51.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.443+0000 7f277e64d640 1 -- 172.21.15.160:0/3153841779 wait complete. 2024-05-09T07:13:51.473 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:13:51.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.574+0000 7f5be357e640 1 Processor -- start 2024-05-09T07:13:51.576 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.574+0000 7f5be357e640 1 -- start start 2024-05-09T07:13:51.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be357e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5bdc153300 0x7f5bdc0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be357e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc1563b0 con 0x7f5bdc0a5fb0 2024-05-09T07:13:51.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be357e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bdc156690 con 0x7f5bdc0c5e20 2024-05-09T07:13:51.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be357e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5bdc0c1630 con 0x7f5bdc153300 2024-05-09T07:13:51.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be0b19640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5bdc0a5fb0 0x7f5bdc0bbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35316/0 (socket says 172.21.15.160:35316) 2024-05-09T07:13:51.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be0b19640 1 -- 172.21.15.160:0/3336479068 learned_addr learned my addr 172.21.15.160:0/3336479068 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3354988225 0 0) 0x7f5bdc1563b0 con 0x7f5bdc0a5fb0 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5be131a640 1 --2- 172.21.15.160:0/3336479068 >> v2:172.21.15.188:3300/0 conn(0x7f5bdc153300 0x7f5bdc0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bc8003520 con 0x7f5bdc0a5fb0 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 568813552 0 0) 0x7f5bc8003520 con 0x7f5bdc0a5fb0 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 >> v2:172.21.15.188:3300/0 conn(0x7f5bdc153300 msgr2=0x7f5bdc0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.575+0000 7f5bd3fff640 1 --2- 172.21.15.160:0/3336479068 >> v2:172.21.15.188:3300/0 conn(0x7f5bdc153300 0x7f5bdc0c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 >> v1:172.21.15.188:6789/0 conn(0x7f5bdc0c5e20 legacy=0x7f5bdc15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc0c1c20 con 0x7f5bdc0a5fb0 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f5bcc001670 con 0x7f5bdc0a5fb0 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 >> v1:172.21.15.160:6789/0 conn(0x7f5bdc0a5fb0 legacy=0x7f5bdc0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bc8003a00 0x7f5bc8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8004800 0x7f5bc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc80051c0 0x7f5bc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc0c1630 con 0x7f5bc8003a00 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc800c0c0 con 0x7f5bc8004800 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5bc800c2c0 con 0x7f5bc80051c0 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be131a640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bc8003a00 0x7f5bc8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be0b19640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8004800 0x7f5bc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be1b1b640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc80051c0 0x7f5bc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be131a640 1 -- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc80051c0 msgr2=0x7f5bc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be131a640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc80051c0 0x7f5bc800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be131a640 1 -- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8004800 msgr2=0x7f5bc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be131a640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8004800 0x7f5bc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be131a640 1 -- 172.21.15.160:0/3336479068 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc8003520 con 0x7f5bc8003a00 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be1b1b640 1 -- 172.21.15.160:0/3336479068 reap_dead start 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.576+0000 7f5be0b19640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bc8004800 0x7f5bc8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:51.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be131a640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bc8003a00 0x7f5bc8003e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009ac0 tx=0x7f5bc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3df1625e6d542cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8036070 con 0x7f5bc8003a00 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd8031400 con 0x7f5bc8003a00 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5bd3fff640 1 -- 172.21.15.160:0/3336479068 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8009690 con 0x7f5bc8003a00 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be357e640 1 -- 172.21.15.160:0/3336479068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bc8003a00 msgr2=0x7f5bc8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be357e640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bc8003a00 0x7f5bc8003e20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009ac0 tx=0x7f5bc8005830 comp rx=0 tx=0).stop 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be357e640 1 -- 172.21.15.160:0/3336479068 shutdown_connections 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be357e640 1 --2- 172.21.15.160:0/3336479068 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bc8003a00 0x7f5bc8003e20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be357e640 1 -- 172.21.15.160:0/3336479068 >> 172.21.15.160:0/3336479068 conn(0x7f5bdc0c7580 msgr2=0x7f5bdc0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.577+0000 7f5be357e640 1 -- 172.21.15.160:0/3336479068 shutdown_connections 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.578+0000 7f5be357e640 1 -- 172.21.15.160:0/3336479068 wait complete. 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 Processor -- start 2024-05-09T07:13:51.583 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 -- start start 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc8003a00 0x7f5bdc0bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 0x7f5bdc0a9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bdc154460 0x7f5bdc0a9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc0c2420 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5bdc0c19d0 con 0x7f5bdc154460 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be357e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5bdc0bd220 con 0x7f5bc8003a00 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be0b19640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 0x7f5bdc0a9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be0b19640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 0x7f5bdc0a9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40174/0 (socket says 172.21.15.160:40174) 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.579+0000 7f5be0b19640 1 -- 172.21.15.160:0/2326667591 learned_addr learned my addr 172.21.15.160:0/2326667591 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:51.586 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be0b19640 1 -- 172.21.15.160:0/2326667591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc8003a00 msgr2=0x7f5bdc0bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be0b19640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc8003a00 0x7f5bdc0bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be0b19640 1 -- 172.21.15.160:0/2326667591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bdc154460 msgr2=0x7f5bdc0a9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be0b19640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bdc154460 0x7f5bdc0a9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be0b19640 1 -- 172.21.15.160:0/2326667591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bdc153900 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be0b19640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 0x7f5bdc0a9990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000c00 tx=0x7f5bcc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00ee60 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bcc018070 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bdc153ba0 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00cab0 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.580+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bdc154160 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.581+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bdc0756c0 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.581+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5bcc00cc50 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.582+0000 7f5bd1ffb640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5bc006be30 0x7f5bc006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.582+0000 7f5be131a640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5bc006be30 0x7f5bc006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.587 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.582+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5bcc08af90 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.582+0000 7f5be131a640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5bc006be30 0x7f5bc006e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8033000 tx=0x7f5bd802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.588 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.586+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5bcc059600 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.794+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bdc0bdab0 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.798 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.795+0000 7f5bd1ffb640 1 -- 172.21.15.160:0/2326667591 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+5183 (secure 0 0 0) 0x7f5bcc058f00 con 0x7f5bdc1ec5c0 2024-05-09T07:13:51.799 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:51.799 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:13:49.846482+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 6 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5bc006be30 msgr2=0x7f5bc006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5bc006be30 0x7f5bc006e2f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8033000 tx=0x7f5bd802e000 comp rx=0 tx=0).stop 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 msgr2=0x7f5bdc0a9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 0x7f5bdc0a9990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc000c00 tx=0x7f5bcc00eb50 comp rx=0 tx=0).stop 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 shutdown_connections 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5bc006be30 0x7f5bc006e2f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5bdc154460 0x7f5bdc0a9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1ec5c0 0x7f5bdc0a9990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 --2- 172.21.15.160:0/2326667591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5bc8003a00 0x7f5bdc0bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 >> 172.21.15.160:0/2326667591 conn(0x7f5bdc0c7580 msgr2=0x7f5bdc0a8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:51.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 shutdown_connections 2024-05-09T07:13:51.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.797+0000 7f5be357e640 1 -- 172.21.15.160:0/2326667591 wait complete. 2024-05-09T07:13:51.810 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:51.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.908+0000 7f13cfc9a640 1 Processor -- start 2024-05-09T07:13:51.910 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.909+0000 7f13cfc9a640 1 -- start start 2024-05-09T07:13:51.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.909+0000 7f13cfc9a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f13c80c8b90 0x7f13c80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.909+0000 7f13cfc9a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c8159840 con 0x7f13c80c5610 2024-05-09T07:13:51.912 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.909+0000 7f13cfc9a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c8159560 con 0x7f13c80be320 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.909+0000 7f13cfc9a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f13c8159310 con 0x7f13c80c8b90 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13cda36640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f13c80c5610 0x7f13c8155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:35332/0 (socket says 172.21.15.160:35332) 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13cda36640 1 -- 172.21.15.160:0/3147915338 learned_addr learned my addr 172.21.15.160:0/3147915338 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13ce237640 1 --2- 172.21.15.160:0/3147915338 >> v2:172.21.15.160:3300/0 conn(0x7f13c80c8b90 0x7f13c80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13cca34640 1 -- 172.21.15.160:0/3147915338 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 447501108 0 0) 0x7f13c8159840 con 0x7f13c80c5610 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13cca34640 1 -- 172.21.15.160:0/3147915338 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13b4003520 con 0x7f13c80c5610 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13ce237640 1 -- 172.21.15.160:0/3147915338 >> v1:172.21.15.188:6789/0 conn(0x7f13c80be320 legacy=0x7f13c8155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13ce237640 1 -- 172.21.15.160:0/3147915338 >> v1:172.21.15.160:6789/0 conn(0x7f13c80c5610 legacy=0x7f13c8155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13cca34640 1 -- 172.21.15.160:0/3147915338 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2036923913 0 0) 0x7f13b4003520 con 0x7f13c80c5610 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.910+0000 7f13ce237640 1 -- 172.21.15.160:0/3147915338 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c80c77d0 con 0x7f13c80c8b90 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13ce237640 1 --2- 172.21.15.160:0/3147915338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80c59f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009d90 tx=0x7f13c002a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e68b23b95783bd17 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cca34640 1 -- 172.21.15.160:0/3147915338 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c4001670 con 0x7f13c80c8b90 2024-05-09T07:13:51.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cca34640 1 -- 172.21.15.160:0/3147915338 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0007540 con 0x7f13c80c8b90 2024-05-09T07:13:51.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cca34640 1 -- 172.21.15.160:0/3147915338 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0007910 con 0x7f13c80c8b90 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cfc9a640 1 -- 172.21.15.160:0/3147915338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 msgr2=0x7f13c80c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/3147915338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80c59f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009d90 tx=0x7f13c002a560 comp rx=0 tx=0).stop 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cfc9a640 1 -- 172.21.15.160:0/3147915338 shutdown_connections 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/3147915338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80c59f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.911+0000 7f13cfc9a640 1 -- 172.21.15.160:0/3147915338 >> 172.21.15.160:0/3147915338 conn(0x7f13c8072e90 msgr2=0x7f13c80ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.912+0000 7f13cfc9a640 1 -- 172.21.15.160:0/3147915338 shutdown_connections 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.912+0000 7f13cfc9a640 1 -- 172.21.15.160:0/3147915338 wait complete. 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 Processor -- start 2024-05-09T07:13:51.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 -- start start 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c80be320 0x7f13c8153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c80c5610 0x7f13c8155ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80bf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80ad6a0 con 0x7f13c80c8b90 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f13c80c7d90 con 0x7f13c80c5610 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13cfc9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f13c8152bb0 con 0x7f13c80be320 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13ce237640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80bf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13ce237640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80bf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:40190/0 (socket says 172.21.15.160:40190) 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.913+0000 7f13ce237640 1 -- 172.21.15.160:0/2410532727 learned_addr learned my addr 172.21.15.160:0/2410532727 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13cd235640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c80c5610 0x7f13c8155ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13cda36640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c80be320 0x7f13c8153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ce237640 1 -- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c80be320 msgr2=0x7f13c8153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ce237640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c80be320 0x7f13c8153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ce237640 1 -- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c80c5610 msgr2=0x7f13c8155ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ce237640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c80c5610 0x7f13c8155ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ce237640 1 -- 172.21.15.160:0/2410532727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c8153760 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ce237640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80bf080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009eb0 tx=0x7f13c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0040070 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0031bc0 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0009450 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c8153a40 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.914+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c81542e0 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.915+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c814e970 con 0x7f13c80c8b90 2024-05-09T07:13:51.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.916+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f13c0039040 con 0x7f13c80c8b90 2024-05-09T07:13:51.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.916+0000 7f13ba7fc640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f13ac06be30 0x7f13ac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:51.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.916+0000 7f13cda36640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f13ac06be30 0x7f13ac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:51.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.916+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f13c00ac9e0 con 0x7f13c80c8b90 2024-05-09T07:13:51.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.916+0000 7f13cda36640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f13ac06be30 0x7f13ac06e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13c4005880 tx=0x7f13c4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:51.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:51.920+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13c814e970 con 0x7f13c80c8b90 2024-05-09T07:13:52.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.095+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c81544e0 con 0x7f13c80c8b90 2024-05-09T07:13:52.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.096+0000 7f13ba7fc640 1 -- 172.21.15.160:0/2410532727 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f13c81544e0 con 0x7f13c80c8b90 2024-05-09T07:13:52.097 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:52.098 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f13ac06be30 msgr2=0x7f13ac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f13ac06be30 0x7f13ac06e2f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f13c4005880 tx=0x7f13c4009040 comp rx=0 tx=0).stop 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 msgr2=0x7f13c80bf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80bf080 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009eb0 tx=0x7f13c0004a40 comp rx=0 tx=0).stop 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 shutdown_connections 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f13ac06be30 0x7f13ac06e2f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f13c80c5610 0x7f13c8155ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f13c80c8b90 0x7f13c80bf080 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 --2- 172.21.15.160:0/2410532727 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f13c80be320 0x7f13c8153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 >> 172.21.15.160:0/2410532727 conn(0x7f13c8072e90 msgr2=0x7f13c81573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 shutdown_connections 2024-05-09T07:13:52.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.098+0000 7f13cfc9a640 1 -- 172.21.15.160:0/2410532727 wait complete. 2024-05-09T07:13:52.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T07:13:44.804208+0000', 'modified': '2024-05-09T07:13:49.846482+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6833/3545719682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 3545719682}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 3545719682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4296, 'qdb_cluster': [4296]} 2024-05-09T07:13:52.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T07:13:52.514 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.613+0000 7f7b2d509640 1 Processor -- start 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.613+0000 7f7b2d509640 1 -- start start 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b2d509640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7b280c36e0 0x7f7b2806ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b2d509640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b2806e230 con 0x7f7b2806de50 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b2d509640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b280c6520 con 0x7f7b280c8640 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b2d509640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f7b280c7540 con 0x7f7b280c36e0 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b267fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7b280c36e0 0x7f7b2806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b26ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f7b2806de50 0x7f7b280c5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36334/0 (socket says 172.21.15.160:36334) 2024-05-09T07:13:52.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b26ffd640 1 -- 172.21.15.160:0/1484644223 learned_addr learned my addr 172.21.15.160:0/1484644223 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b267fc640 1 -- 172.21.15.160:0/1484644223 >> v1:172.21.15.188:6789/0 conn(0x7f7b280c8640 legacy=0x7f7b280c8c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b267fc640 1 -- 172.21.15.160:0/1484644223 >> v1:172.21.15.160:6789/0 conn(0x7f7b2806de50 legacy=0x7f7b280c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b267fc640 1 -- 172.21.15.160:0/1484644223 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b280c4700 con 0x7f7b280c36e0 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.614+0000 7f7b25ffb640 1 -- 172.21.15.160:0/1484644223 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1469047577 0 0) 0x7f7b2806e230 con 0x7f7b2806de50 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.615+0000 7f7b267fc640 1 --2- 172.21.15.160:0/1484644223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c36e0 0x7f7b2806ced0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0097a0 tx=0x7f7b1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d5bd48f9a56b2d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.615+0000 7f7b25ffb640 1 -- 172.21.15.160:0/1484644223 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c037070 con 0x7f7b280c36e0 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.615+0000 7f7b25ffb640 1 -- 172.21.15.160:0/1484644223 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b1c032440 con 0x7f7b280c36e0 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.615+0000 7f7b25ffb640 1 -- 172.21.15.160:0/1484644223 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1c032810 con 0x7f7b280c36e0 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 -- 172.21.15.160:0/1484644223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c36e0 msgr2=0x7f7b2806ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 --2- 172.21.15.160:0/1484644223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c36e0 0x7f7b2806ced0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c0097a0 tx=0x7f7b1c02a1e0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 -- 172.21.15.160:0/1484644223 shutdown_connections 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 --2- 172.21.15.160:0/1484644223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c36e0 0x7f7b2806ced0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 -- 172.21.15.160:0/1484644223 >> 172.21.15.160:0/1484644223 conn(0x7f7b28066f90 msgr2=0x7f7b28067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 -- 172.21.15.160:0/1484644223 shutdown_connections 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.616+0000 7f7b2d509640 1 -- 172.21.15.160:0/1484644223 wait complete. 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.617+0000 7f7b2d509640 1 Processor -- start 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.617+0000 7f7b2d509640 1 -- start start 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b2d509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b2806de50 0x7f7b281e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b2d509640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b280c36e0 0x7f7b281eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b2d509640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 0x7f7b280ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b2d509640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280c5980 con 0x7f7b280c8640 2024-05-09T07:13:52.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b2d509640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b280c44b0 con 0x7f7b280c36e0 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b2d509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b280c4cc0 con 0x7f7b2806de50 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 0x7f7b280ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 0x7f7b280ace10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58328/0 (socket says 172.21.15.160:58328) 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 -- 172.21.15.160:0/3798694957 learned_addr learned my addr 172.21.15.160:0/3798694957 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b267fc640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b280c36e0 0x7f7b281eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b2806de50 0x7f7b281e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 -- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b2806de50 msgr2=0x7f7b281e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b2806de50 0x7f7b281e21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 -- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b280c36e0 msgr2=0x7f7b281eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b280c36e0 0x7f7b281eed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 -- 172.21.15.160:0/3798694957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b281ef270 con 0x7f7b280c8640 2024-05-09T07:13:52.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b277fe640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 0x7f7b280ace10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c3c0 tx=0x7f7b2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.618+0000 7f7b0f7fe640 1 -- 172.21.15.160:0/3798694957 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20017070 con 0x7f7b280c8640 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.619+0000 7f7b0f7fe640 1 -- 172.21.15.160:0/3798694957 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b200035f0 con 0x7f7b280c8640 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.619+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b280c2d80 con 0x7f7b280c8640 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.619+0000 7f7b0f7fe640 1 -- 172.21.15.160:0/3798694957 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b200039e0 con 0x7f7b280c8640 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.619+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281ee530 con 0x7f7b280c8640 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.620+0000 7f7b0f7fe640 1 -- 172.21.15.160:0/3798694957 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7b2001a070 con 0x7f7b280c8640 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.620+0000 7f7b0f7fe640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7b1406be60 0x7f7b1406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.621+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7b1406be60 0x7f7b1406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.621+0000 7f7b0f7fe640 1 -- 172.21.15.160:0/3798694957 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7b2008a540 con 0x7f7b280c8640 2024-05-09T07:13:52.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.621+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7b1406be60 0x7f7b1406e320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b180012e0 tx=0x7f7b18005eb0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.621+0000 7f7b3020e740 1 Processor -- start 2024-05-09T07:13:52.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.621+0000 7f7b3020e740 1 -- start start 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd4968b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 0x55dd496811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49667990 0x55dd496816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b3020e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dd495a94d0 con 0x55dd4968acf0 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd495513e0 con 0x55dd49667990 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55dd495a5490 con 0x55dd49684630 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd4968b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd4968b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58340/0 (socket says 172.21.15.160:58340) 2024-05-09T07:13:52.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 -- 172.21.15.160:0/2430916413 learned_addr learned my addr 172.21.15.160:0/2430916413 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 -- 172.21.15.160:0/2430916413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 msgr2=0x55dd496811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 0x55dd496811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 -- 172.21.15.160:0/2430916413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49667990 msgr2=0x55dd496816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49667990 0x55dd496816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.622+0000 7f7b26ffd640 1 -- 172.21.15.160:0/2430916413 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd492bd2d0 con 0x55dd4968acf0 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd4968b0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b18001c40 tx=0x7f7b18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bdd2e4a1cd4e42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b0cff9640 1 -- 172.21.15.160:0/2430916413 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180159c0 con 0x55dd4968acf0 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b0cff9640 1 -- 172.21.15.160:0/2430916413 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b18015b60 con 0x55dd4968acf0 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b0cff9640 1 -- 172.21.15.160:0/2430916413 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18021450 con 0x55dd4968acf0 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 -- 172.21.15.160:0/2430916413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 msgr2=0x55dd4968b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd4968b0d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7b18001c40 tx=0x7f7b18012000 comp rx=0 tx=0).stop 2024-05-09T07:13:52.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 -- 172.21.15.160:0/2430916413 shutdown_connections 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49667990 0x55dd496816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 0x55dd496811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 --2- 172.21.15.160:0/2430916413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd4968b0d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 -- 172.21.15.160:0/2430916413 >> 172.21.15.160:0/2430916413 conn(0x55dd49642b60 msgr2=0x55dd49667280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 -- 172.21.15.160:0/2430916413 shutdown_connections 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.623+0000 7f7b3020e740 1 -- 172.21.15.160:0/2430916413 wait complete. 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.624+0000 7f7b3020e740 1 -- start start 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.624+0000 7f7b3020e740 1 Processor -- start 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd49692510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd4968acf0 0x55dd496634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd49689df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dd49641d60 con 0x55dd49684630 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd495648e0 con 0x55dd49689970 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55dd4968b610 con 0x55dd4968acf0 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 10 client.0 _ll_register_callbacks cb 0x55dd495319c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd49692510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd49692510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58342/0 (socket says 172.21.15.160:58342) 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 -- 172.21.15.160:0/396902661 learned_addr learned my addr 172.21.15.160:0/396902661 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b267fc640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd4968acf0 0x55dd496634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b277fe640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd49689df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 -- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd4968acf0 msgr2=0x55dd496634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd4968acf0 0x55dd496634f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 -- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 msgr2=0x55dd49689df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd49689df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 -- 172.21.15.160:0/396902661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd493876b0 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd49692510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b18015810 tx=0x7f7b18001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43f58812c99c1927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b05ffb640 1 -- 172.21.15.160:0/396902661 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180136f0 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b05ffb640 1 -- 172.21.15.160:0/396902661 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7b18010070 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.625+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b18015b60 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b05ffb640 1 -- 172.21.15.160:0/396902661 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f7b18032050 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 10 client.4449 fetch_fsmap learned FSMap version 6 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55dd49689450 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b05ffb640 1 -- 172.21.15.160:0/396902661 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b18015b60 con 0x55dd49684630 2024-05-09T07:13:52.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b05ffb640 1 -- 172.21.15.160:0/396902661 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f7b18035020 con 0x55dd49684630 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 10 client.4449 fetch_fsmap finished waiting for FSMap version 6 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 10 client.4449 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 20 client.4449 populate_metadata read hostname 'smithi160' 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd49662950 0x55dd496a1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 4 client.4449 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 --> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd495b6c00 con 0x55dd49662950 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.626+0000 7f7b267fc640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd49662950 0x55dd496a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b05ffb640 10 client.4449 ms_handle_connect on v2:172.21.15.188:6832/3545719682 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b267fc640 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd49662950 0x55dd496a1160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=272667adf828adc1 server_cookie=940f5bcfa86e1be5 in_seq=0 out_seq=0 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b05ffb640 1 -- 172.21.15.160:0/396902661 <== mds.0 v2:172.21.15.188:6832/3545719682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd495b6c00 con 0x55dd49662950 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b05ffb640 10 client.4449 handle_command_reply: tid=0 2024-05-09T07:13:52.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b3020e740 1 client.4449 shutdown 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 msgr2=0x55dd49692510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.627+0000 7f7b3020e740 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd49692510 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b18015810 tx=0x7f7b18001250 comp rx=0 tx=0).stop 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 shutdown_connections 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd49662950 0x55dd496a1160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd49689df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd4968acf0 0x55dd496634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b05ffb640 0 client.4449 ms_handle_reset on v2:172.21.15.188:6832/3545719682 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 --2- 172.21.15.160:0/396902661 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd49692510 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 >> 172.21.15.160:0/396902661 conn(0x55dd49667990 msgr2=0x55dd49681190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 shutdown_connections 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 1 -- 172.21.15.160:0/396902661 wait complete. 2024-05-09T07:13:52.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.628+0000 7f7b3020e740 20 client.4449 trim_cache size 0 max 16384 2024-05-09T07:13:52.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.645+0000 7f7b3020e740 1 Processor -- start 2024-05-09T07:13:52.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.645+0000 7f7b3020e740 1 -- start start 2024-05-09T07:13:52.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 0x55dd4967ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd496832a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd496837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b3020e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281f8020 con 0x55dd4968acf0 2024-05-09T07:13:52.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281f7460 con 0x55dd49689970 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7b281f79d0 con 0x55dd49684630 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd496837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd496837e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58350/0 (socket says 172.21.15.160:58350) 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 -- 172.21.15.160:0/3628692212 learned_addr learned my addr 172.21.15.160:0/3628692212 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 -- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 msgr2=0x55dd4967ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 0x55dd4967ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 -- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 msgr2=0x55dd496832a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b267fc640 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd496832a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd496832a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.646+0000 7f7b277fe640 1 -- 172.21.15.160:0/3628692212 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd49641f90 con 0x55dd4968acf0 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b277fe640 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd496837e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b20060b60 tx=0x7f7b200010c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44adad98ba3e935 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b07fff640 1 -- 172.21.15.160:0/3628692212 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20061090 con 0x55dd4968acf0 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b07fff640 1 -- 172.21.15.160:0/3628692212 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b200567b0 con 0x55dd4968acf0 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b07fff640 1 -- 172.21.15.160:0/3628692212 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2008a0c0 con 0x55dd4968acf0 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 -- 172.21.15.160:0/3628692212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 msgr2=0x55dd496837e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd496837e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7b20060b60 tx=0x7f7b200010c0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 -- 172.21.15.160:0/3628692212 shutdown_connections 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd49689970 0x55dd496832a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd4968acf0 0x55dd496837e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 --2- 172.21.15.160:0/3628692212 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49684630 0x55dd4967ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 -- 172.21.15.160:0/3628692212 >> 172.21.15.160:0/3628692212 conn(0x55dd49667990 msgr2=0x55dd49667260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 -- 172.21.15.160:0/3628692212 shutdown_connections 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.647+0000 7f7b3020e740 1 -- 172.21.15.160:0/3628692212 wait complete. 2024-05-09T07:13:52.656 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 -- start start 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 Processor -- start 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd496fa270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49689970 0x55dd49699060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd4968acf0 0x55dd496fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55dd4968a3d0 con 0x55dd49684630 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55dd49688850 con 0x55dd4968acf0 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55dd49661ad0 con 0x55dd49689970 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b3020e740 10 client.0 _ll_register_callbacks cb 0x7f7b18001b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b26ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd496fa270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.648+0000 7f7b26ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd496fa270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58354/0 (socket says 172.21.15.160:58354) 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 -- 172.21.15.160:0/4091348626 learned_addr learned my addr 172.21.15.160:0/4091348626 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:52.657 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b267fc640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49689970 0x55dd49699060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 -- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49689970 msgr2=0x55dd49699060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49689970 0x55dd49699060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b277fe640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd4968acf0 0x55dd496fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 -- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd4968acf0 msgr2=0x55dd496fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd4968acf0 0x55dd496fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 -- 172.21.15.160:0/4091348626 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4965dda0 con 0x55dd49684630 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b267fc640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49689970 0x55dd49699060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b277fe640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd4968acf0 0x55dd496fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b26ffd640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd496fa270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b180153a0 tx=0x7f7b18001400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb101e6c323dd278 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b05ffb640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b05ffb640 1 -- 172.21.15.160:0/4091348626 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b180136f0 con 0x55dd49684630 2024-05-09T07:13:52.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b05ffb640 1 -- 172.21.15.160:0/4091348626 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7b18028040 con 0x55dd49684630 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b200567b0 con 0x55dd49684630 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b05ffb640 1 -- 172.21.15.160:0/4091348626 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) ==== 24+0+0 (secure 0 0 0) 0x7f7b18042050 con 0x55dd49684630 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 10 client.4455 fetch_fsmap learned FSMap version 6 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) -- 0x55dd496639e0 con 0x55dd49684630 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b05ffb640 1 -- 172.21.15.160:0/4091348626 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b200567b0 con 0x55dd49684630 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b05ffb640 1 -- 172.21.15.160:0/4091348626 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 6) ==== 3502+0+0 (secure 0 0 0) 0x7f7b18045020 con 0x55dd49684630 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 10 client.4455 fetch_fsmap finished waiting for FSMap version 6 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 10 client.4455 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 20 client.4455 populate_metadata read hostname 'smithi160' 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd496a5540 0x55dd496a5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 4 client.4455 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T07:13:52.659 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.649+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 --> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] -- command(tid 0: {"prefix": "status"}) -- 0x55dd495b0f50 con 0x55dd496a5540 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b267fc640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd496a5540 0x55dd496a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b05ffb640 10 client.4455 ms_handle_connect on v2:172.21.15.188:6832/3545719682 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b267fc640 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd496a5540 0x55dd496a5920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8e572da1c1c2ffc server_cookie=866b949a68265996 in_seq=0 out_seq=0 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b05ffb640 1 -- 172.21.15.160:0/4091348626 <== mds.0 v2:172.21.15.188:6832/3545719682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55dd495b0f50 con 0x55dd496a5540 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b05ffb640 10 client.4455 handle_command_reply: tid=0 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 client.4455 shutdown 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 msgr2=0x55dd496fa270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd496fa270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7b180153a0 tx=0x7f7b18001400 comp rx=0 tx=0).stop 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 shutdown_connections 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55dd496a5540 0x55dd496a5920 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55dd4968acf0 0x55dd496fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.660 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55dd49689970 0x55dd49699060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b05ffb640 0 client.4455 ms_handle_reset on v2:172.21.15.188:6832/3545719682 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 --2- 172.21.15.160:0/4091348626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55dd49684630 0x55dd496fa270 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 >> 172.21.15.160:0/4091348626 conn(0x55dd49667990 msgr2=0x55dd49677fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 shutdown_connections 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 1 -- 172.21.15.160:0/4091348626 wait complete. 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.650+0000 7f7b3020e740 20 client.4455 trim_cache size 0 max 16384 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "22e9edb5-902d-460d-aabf-1eff035c9624", 2024-05-09T07:13:52.661 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 0, 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 4296, 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 7.1781617789999999, 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 6, 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 25, 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 24, 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 13.832442645 2024-05-09T07:13:52.662 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7b1406be60 msgr2=0x7f7b1406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7b1406be60 0x7f7b1406e320 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b180012e0 tx=0x7f7b18005eb0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 msgr2=0x7f7b280ace10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 0x7f7b280ace10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c3c0 tx=0x7f7b2000c880 comp rx=0 tx=0).stop 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 shutdown_connections 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7b1406be60 0x7f7b1406e320 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7b280c36e0 0x7f7b281eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b280c8640 0x7f7b280ace10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.664 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 --2- 172.21.15.160:0/3798694957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7b2806de50 0x7f7b281e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 >> 172.21.15.160:0/3798694957 conn(0x7f7b28066f90 msgr2=0x7f7b2806cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 shutdown_connections 2024-05-09T07:13:52.665 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.651+0000 7f7b2d509640 1 -- 172.21.15.160:0/3798694957 wait complete. 2024-05-09T07:13:52.665 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T07:13:52.665 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:52.665 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T07:13:52.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.763+0000 7fcd3f015640 1 Processor -- start 2024-05-09T07:13:52.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.763+0000 7fcd3f015640 1 -- start start 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd3f015640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd38157510 0x7fcd380c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd3f015640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd380bea40 con 0x7fcd380c7200 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd3f015640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd380bed20 con 0x7fcd380c8110 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd3f015640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fcd380bf000 con 0x7fcd38157510 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd3cdb1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fcd38157510 0x7fcd380c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd37fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fcd380c7200 0x7fcd380c5920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36336/0 (socket says 172.21.15.160:36336) 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.764+0000 7fcd37fff640 1 -- 172.21.15.160:0/3846830617 learned_addr learned my addr 172.21.15.160:0/3846830617 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd3cdb1640 1 -- 172.21.15.160:0/3846830617 >> v1:172.21.15.188:6789/0 conn(0x7fcd380c8110 legacy=0x7fcd380c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd3cdb1640 1 -- 172.21.15.160:0/3846830617 >> v1:172.21.15.160:6789/0 conn(0x7fcd380c7200 legacy=0x7fcd380c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.768 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd377fe640 1 -- 172.21.15.160:0/3846830617 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 759023567 0 0) 0x7fcd380bea40 con 0x7fcd380c7200 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd3cdb1640 1 -- 172.21.15.160:0/3846830617 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd380bf2e0 con 0x7fcd38157510 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd3cdb1640 1 --2- 172.21.15.160:0/3846830617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd38157510 0x7fcd380c7bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcd280097a0 tx=0x7fcd2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224dd6393f1fbdf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd377fe640 1 -- 172.21.15.160:0/3846830617 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28037070 con 0x7fcd38157510 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd377fe640 1 -- 172.21.15.160:0/3846830617 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd28032440 con 0x7fcd38157510 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.765+0000 7fcd377fe640 1 -- 172.21.15.160:0/3846830617 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28032810 con 0x7fcd38157510 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 -- 172.21.15.160:0/3846830617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd38157510 msgr2=0x7fcd380c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 --2- 172.21.15.160:0/3846830617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd38157510 0x7fcd380c7bd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fcd280097a0 tx=0x7fcd2802a1a0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 -- 172.21.15.160:0/3846830617 shutdown_connections 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 --2- 172.21.15.160:0/3846830617 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd38157510 0x7fcd380c7bd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 -- 172.21.15.160:0/3846830617 >> 172.21.15.160:0/3846830617 conn(0x7fcd38066f90 msgr2=0x7fcd38067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 -- 172.21.15.160:0/3846830617 shutdown_connections 2024-05-09T07:13:52.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.766+0000 7fcd3f015640 1 -- 172.21.15.160:0/3846830617 wait complete. 2024-05-09T07:13:52.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.767+0000 7fcd3f015640 1 Processor -- start 2024-05-09T07:13:52.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.767+0000 7fcd3f015640 1 -- start start 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3f015640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd380c7200 0x7fcd381eb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3f015640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 0x7fcd380c2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3f015640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd38157510 0x7fcd380c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3f015640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcd380a8880 con 0x7fcd380c8110 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3f015640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcd380a8700 con 0x7fcd38157510 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3f015640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcd380bfb30 con 0x7fcd380c7200 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 0x7fcd380c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 0x7fcd380c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58376/0 (socket says 172.21.15.160:58376) 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 -- 172.21.15.160:0/1499627151 learned_addr learned my addr 172.21.15.160:0/1499627151 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3d5b2640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd38157510 0x7fcd380c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.773 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3cdb1640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd380c7200 0x7fcd381eb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 -- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd380c7200 msgr2=0x7fcd381eb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd380c7200 0x7fcd381eb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 -- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd38157510 msgr2=0x7fcd380c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd38157510 0x7fcd380c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 -- 172.21.15.160:0/1499627151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd380c0b20 con 0x7fcd380c8110 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd3d5b2640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd38157510 0x7fcd380c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.768+0000 7fcd37fff640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 0x7fcd380c2450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c8c0 tx=0x7fcd2c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.769+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c018070 con 0x7fcd380c8110 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.769+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd2c00a640 con 0x7fcd380c8110 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.769+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd381ebaf0 con 0x7fcd380c8110 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.769+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00aa70 con 0x7fcd380c8110 2024-05-09T07:13:52.774 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.769+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd381ebdd0 con 0x7fcd380c8110 2024-05-09T07:13:52.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.770+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd380756c0 con 0x7fcd380c8110 2024-05-09T07:13:52.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.770+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcd2c01b070 con 0x7fcd380c8110 2024-05-09T07:13:52.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.771+0000 7fcd357fa640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcd2406be30 0x7fcd2406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:52.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.771+0000 7fcd3cdb1640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcd2406be30 0x7fcd2406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:52.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.771+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcd2c08a7a0 con 0x7fcd380c8110 2024-05-09T07:13:52.775 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.771+0000 7fcd3cdb1640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcd2406be30 0x7fcd2406e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd2802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:52.776 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.775+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcd2c058ea0 con 0x7fcd380c8110 2024-05-09T07:13:52.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.986+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fcd380ac720 con 0x7fcd380c8110 2024-05-09T07:13:52.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.987+0000 7fcd357fa640 1 -- 172.21.15.160:0/1499627151 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) ==== 53+0+88 (secure 0 0 0) 0x7fcd2c0587a0 con 0x7fcd380c8110 2024-05-09T07:13:52.988 INFO:teuthology.orchestra.run.smithi160.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcd2406be30 msgr2=0x7fcd2406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcd2406be30 0x7fcd2406e2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd28034000 tx=0x7fcd2802e000 comp rx=0 tx=0).stop 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 msgr2=0x7fcd380c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 0x7fcd380c2450 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c8c0 tx=0x7fcd2c00cd80 comp rx=0 tx=0).stop 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 shutdown_connections 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcd2406be30 0x7fcd2406e2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcd38157510 0x7fcd380c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcd380c8110 0x7fcd380c2450 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 --2- 172.21.15.160:0/1499627151 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcd380c7200 0x7fcd381eb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 >> 172.21.15.160:0/1499627151 conn(0x7fcd38066f90 msgr2=0x7fcd380a99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 shutdown_connections 2024-05-09T07:13:52.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:52.989+0000 7fcd3f015640 1 -- 172.21.15.160:0/1499627151 wait complete. 2024-05-09T07:13:53.001 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-09T07:13:53.001 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T07:13:53.001 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2024-05-09T07:13:53.001 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-09T07:13:53.001 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:53.001 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-09T07:13:53.002 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T07:13:53.002 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T07:13:53.002 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T07:13:53.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:53.002 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:13:53.018 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f0 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f1 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42768sec preferred_lft 42768sec 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2024-05-09T07:13:53.019 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:13:53.020 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add name ceph-brx type bridge 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr flush dev ceph-brx 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set ceph-brx up 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T07:13:53.020 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:13:53.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:13:53.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:53.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:53.235 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T07:13:53.264 INFO:teuthology.orchestra.run.smithi160.stdout:1 2024-05-09T07:13:53.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:53.266 DEBUG:teuthology.orchestra.run.smithi160:> route 2024-05-09T07:13:53.329 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2024-05-09T07:13:53.329 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T07:13:53.329 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T07:13:53.329 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T07:13:53.329 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T07:13:53.330 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:13:53.330 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:13:53.330 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T07:13:53.330 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T07:13:53.330 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T07:13:53.330 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:13:53.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:13:53.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:53.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:53.630 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-05-09T07:13:53.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:53.649 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2024-05-09T07:13:53.706 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:13:53.706 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:13:53.706 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T07:13:53.706 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-09T07:13:53.706 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:13:53.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:13:53.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:53.827 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-09T07:13:53.827 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-09T07:13:53.828 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:13:53.877 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:13:53.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:53.961 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:13:53.962 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:13:53.962 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.0 up 2024-05-09T07:13:53.962 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T07:13:53.962 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:13:54.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:13:54.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:54.043 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-09T07:13:54.043 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T07:13:54.043 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:54.058 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-09T07:13:54.059 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T07:13:54.059 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:54.114 DEBUG:teuthology.orchestra.run.smithi160:> sudo modprobe fuse 2024-05-09T07:13:54.180 DEBUG:teuthology.orchestra.run.smithi160:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T07:13:54.236 INFO:teuthology.orchestra.run.smithi160.stdout:/proc 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/sys 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/dev 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/security 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/shm 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/pts 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/run 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/cgroup 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/pstore 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/bpf 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/ 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/selinux 2024-05-09T07:13:54.237 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/hugepages 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/mqueue 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/debug 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/tracing 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/fuse/connections 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/config 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/1000 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T07:13:54.238 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T07:13:54.239 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T07:13:54.239 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T07:13:54.239 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns 2024-05-09T07:13:54.239 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T07:13:54.239 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T07:13:54.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:54.239 DEBUG:teuthology.orchestra.run.smithi160:> ls /sys/fs/fuse/connections 2024-05-09T07:13:54.292 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T07:13:54.293 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-09T07:13:54.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo modprobe fuse 2024-05-09T07:13:54.360 DEBUG:teuthology.orchestra.run.smithi160:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T07:13:54.417 INFO:teuthology.orchestra.run.smithi160.stdout:/proc 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/sys 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/dev 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/security 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/shm 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/pts 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/run 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/cgroup 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/pstore 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/bpf 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/ 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/selinux 2024-05-09T07:13:54.418 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/hugepages 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/mqueue 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/debug 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/tracing 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/fuse/connections 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/config 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/1000 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T07:13:54.419 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T07:13:54.420 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T07:13:54.420 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns 2024-05-09T07:13:54.420 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T07:13:54.420 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T07:13:54.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:54.420 DEBUG:teuthology.orchestra.run.smithi160:> ls /sys/fs/fuse/connections 2024-05-09T07:13:54.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi160.stderr:2024-05-09T07:13:54.460+0000 7f16052e7480 -1 init, newargv = 0x555daa1757c0 newargc=15 2024-05-09T07:13:54.462 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi160.stderr:ceph-fuse[43029]: starting ceph client 2024-05-09T07:13:54.468 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi160.stderr:ceph-fuse[43029]: starting fuse 2024-05-09T07:13:54.477 INFO:teuthology.orchestra.run.smithi160.stdout:40 2024-05-09T07:13:54.478 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> sudo stdin-killer -- python3 -c ' 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> import glob 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> import re 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> import os 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> import subprocess 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> def _find_admin_socket(client_name): 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> files = glob.glob(asok_path) 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-09T07:13:54.478 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> # Given a non-glob path, it better be there 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> if "*" not in asok_path: 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> assert(len(files) == 1) 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> return files[0] 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> for f in files: 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> contents = proc_f.read() 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> if mountpoint in contents: 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> return f 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> print(_find_admin_socket("client.admin")) 2024-05-09T07:13:54.479 DEBUG:teuthology.orchestra.run.smithi160:> ' 2024-05-09T07:13:54.598 INFO:teuthology.orchestra.run.smithi160.stdout:/var/run/ceph/ceph-client.admin.43029.asok 2024-05-09T07:13:54.602 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:54.609 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.43029.asok 2024-05-09T07:13:54.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:54.609 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.43029.asok status 2024-05-09T07:13:54.745 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:13:54.745 INFO:teuthology.orchestra.run.smithi160.stdout: "metadata": { 2024-05-09T07:13:54.745 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T07:13:54.745 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T07:13:54.745 INFO:teuthology.orchestra.run.smithi160.stdout: "entity_id": "admin", 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": "smithi160", 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "pid": "43029", 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "root": "/" 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "dentry_count": 0, 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "dentry_pinned_count": 0, 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 4467, 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "inst": { 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "name": { 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "client", 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "num": 4467 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:13:54.746 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": { 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "192.168.0.1:0", 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 1661283800 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": { 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "192.168.0.1:0", 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 1661283800 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "inst_str": "client.4467 192.168.0.1:0/1661283800", 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "addr_str": "192.168.0.1:0/1661283800", 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "inode_count": 1, 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_epoch": 6, 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_epoch": 25, 2024-05-09T07:13:54.747 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_epoch_barrier": 0, 2024-05-09T07:13:54.748 INFO:teuthology.orchestra.run.smithi160.stdout: "blocklisted": false, 2024-05-09T07:13:54.748 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs" 2024-05-09T07:13:54.748 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:13:54.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:54.753 DEBUG:teuthology.orchestra.run.smithi160:> stat --file-system '--printf=%T 2024-05-09T07:13:54.753 DEBUG:teuthology.orchestra.run.smithi160:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:54.769 INFO:teuthology.orchestra.run.smithi160.stdout:fuseblk 2024-05-09T07:13:54.769 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:54.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:54.769 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:54.836 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-09T07:13:54.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T07:13:54.942 INFO:teuthology.orchestra.run.smithi160.stdout:# file: . 2024-05-09T07:13:54.942 INFO:teuthology.orchestra.run.smithi160.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-09T07:13:54.942 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:54.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:13:54.950 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:54.950 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:13:54.966 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:55.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.105+0000 7ff458cda640 1 Processor -- start 2024-05-09T07:13:55.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.105+0000 7ff458cda640 1 -- start start 2024-05-09T07:13:55.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff458cda640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff454153300 0x7ff4540c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff458cda640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540c5e20 0x7ff4540bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff458cda640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4541563b0 con 0x7ff4540a5fb0 2024-05-09T07:13:55.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff458cda640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ff4540c1630 con 0x7ff454153300 2024-05-09T07:13:55.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff458cda640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4540c19d0 con 0x7ff4540c5e20 2024-05-09T07:13:55.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff452d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540c5e20 0x7ff4540bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff45259c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ff454153300 0x7ff4540c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff451d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff4540a5fb0 0x7ff4540bbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:58076/0 (socket says 172.21.15.160:58076) 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff451d9b640 1 -- 172.21.15.160:0/2844907062 learned_addr learned my addr 172.21.15.160:0/2844907062 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff45159a640 1 -- 172.21.15.160:0/2844907062 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2136746112 0 0) 0x7ff4541563b0 con 0x7ff4540a5fb0 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff45159a640 1 -- 172.21.15.160:0/2844907062 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff43c005730 con 0x7ff4540a5fb0 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff452d9d640 1 -- 172.21.15.160:0/2844907062 >> v2:172.21.15.188:3300/0 conn(0x7ff454153300 msgr2=0x7ff4540c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff452d9d640 1 --2- 172.21.15.160:0/2844907062 >> v2:172.21.15.188:3300/0 conn(0x7ff454153300 0x7ff4540c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff452d9d640 1 -- 172.21.15.160:0/2844907062 >> v1:172.21.15.188:6789/0 conn(0x7ff4540a5fb0 legacy=0x7ff4540bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff452d9d640 1 -- 172.21.15.160:0/2844907062 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff454156690 con 0x7ff4540c5e20 2024-05-09T07:13:55.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.106+0000 7ff45259c640 1 --2- 172.21.15.160:0/2844907062 >> v2:172.21.15.188:3300/0 conn(0x7ff454153300 0x7ff4540c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.107+0000 7ff452d9d640 1 --2- 172.21.15.160:0/2844907062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540c5e20 0x7ff4540bc530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009d90 tx=0x7ff44c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0d45ee329f54f5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.107+0000 7ff45159a640 1 -- 172.21.15.160:0/2844907062 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c007740 con 0x7ff4540c5e20 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.107+0000 7ff45159a640 1 -- 172.21.15.160:0/2844907062 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff44c0078e0 con 0x7ff4540c5e20 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.107+0000 7ff45159a640 1 -- 172.21.15.160:0/2844907062 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44c007c90 con 0x7ff4540c5e20 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 -- 172.21.15.160:0/2844907062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540c5e20 msgr2=0x7ff4540bc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 --2- 172.21.15.160:0/2844907062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540c5e20 0x7ff4540bc530 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff44c009d90 tx=0x7ff44c02a660 comp rx=0 tx=0).stop 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 -- 172.21.15.160:0/2844907062 shutdown_connections 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 --2- 172.21.15.160:0/2844907062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540c5e20 0x7ff4540bc530 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 --2- 172.21.15.160:0/2844907062 >> v2:172.21.15.188:3300/0 conn(0x7ff454153300 0x7ff4540c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 -- 172.21.15.160:0/2844907062 >> 172.21.15.160:0/2844907062 conn(0x7ff4540c7580 msgr2=0x7ff4540a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 -- 172.21.15.160:0/2844907062 shutdown_connections 2024-05-09T07:13:55.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.108+0000 7ff458cda640 1 -- 172.21.15.160:0/2844907062 wait complete. 2024-05-09T07:13:55.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.109+0000 7ff458cda640 1 Processor -- start 2024-05-09T07:13:55.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.109+0000 7ff458cda640 1 -- start start 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff458cda640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540a5fb0 0x7ff4541e66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff458cda640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 0x7ff454152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff458cda640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454153300 0x7ff4541ea530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff458cda640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540c2420 con 0x7ff4540c5e20 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff458cda640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540c1f00 con 0x7ff454153300 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff458cda640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff4540bd220 con 0x7ff4540a5fb0 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 0x7ff454152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 0x7ff454152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58382/0 (socket says 172.21.15.160:58382) 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 -- 172.21.15.160:0/4097824253 learned_addr learned my addr 172.21.15.160:0/4097824253 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff45259c640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540a5fb0 0x7ff4541e66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff452d9d640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454153300 0x7ff4541ea530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 -- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540a5fb0 msgr2=0x7ff4541e66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540a5fb0 0x7ff4541e66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 -- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454153300 msgr2=0x7ff4541ea530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454153300 0x7ff4541ea530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.110+0000 7ff451d9b640 1 -- 172.21.15.160:0/4097824253 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4540ab9b0 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.111+0000 7ff451d9b640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 0x7ff454152380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c3c0 tx=0x7ff44800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.111+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff448017070 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.111+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff448005ce0 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.111+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4540a89b0 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.111+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4480072f0 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.111+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4540ab3f0 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.112+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4540756c0 con 0x7ff4540c5e20 2024-05-09T07:13:55.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.112+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff44801a070 con 0x7ff4540c5e20 2024-05-09T07:13:55.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.113+0000 7ff442ffd640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff43406be30 0x7ff43406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.113+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff44808a4e0 con 0x7ff4540c5e20 2024-05-09T07:13:55.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.113+0000 7ff45259c640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff43406be30 0x7ff43406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.113+0000 7ff45259c640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff43406be30 0x7ff43406e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff444009560 tx=0x7ff444009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.117+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff448058510 con 0x7ff4540c5e20 2024-05-09T07:13:55.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.294+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4540bdab0 con 0x7ff4540c5e20 2024-05-09T07:13:55.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.295+0000 7ff442ffd640 1 -- 172.21.15.160:0/4097824253 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7ff44805c810 con 0x7ff4540c5e20 2024-05-09T07:13:55.298 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:55.298 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:55.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.297+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff43406be30 msgr2=0x7ff43406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.297+0000 7ff458cda640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff43406be30 0x7ff43406e2f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff444009560 tx=0x7ff444009190 comp rx=0 tx=0).stop 2024-05-09T07:13:55.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.297+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 msgr2=0x7ff454152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.297+0000 7ff458cda640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 0x7ff454152380 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff44800c3c0 tx=0x7ff44800c880 comp rx=0 tx=0).stop 2024-05-09T07:13:55.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 shutdown_connections 2024-05-09T07:13:55.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff43406be30 0x7ff43406e2f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff454153300 0x7ff4541ea530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff4540c5e20 0x7ff454152380 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 --2- 172.21.15.160:0/4097824253 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff4540a5fb0 0x7ff4541e66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 >> 172.21.15.160:0/4097824253 conn(0x7ff4540c7580 msgr2=0x7ff45406b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:55.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 shutdown_connections 2024-05-09T07:13:55.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.298+0000 7ff458cda640 1 -- 172.21.15.160:0/4097824253 wait complete. 2024-05-09T07:13:55.310 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:13:55.311 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:13:55.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.412+0000 7fd573dce640 1 Processor -- start 2024-05-09T07:13:55.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.413+0000 7fd573dce640 1 -- start start 2024-05-09T07:13:55.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.413+0000 7fd573dce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd56c06cd20 0x7fd56c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.413+0000 7fd573dce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c0c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.413+0000 7fd573dce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd56c0c3090 con 0x7fd56c0c4df0 2024-05-09T07:13:55.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.413+0000 7fd573dce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fd56c0c1550 con 0x7fd56c06cd20 2024-05-09T07:13:55.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.413+0000 7fd573dce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd56c0c1830 con 0x7fd56c0c2bd0 2024-05-09T07:13:55.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571b6a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd56c06cd20 0x7fd56c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd57236b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd56c0c4df0 0x7fd56c14ece0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36350/0 (socket says 172.21.15.160:36350) 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd57236b640 1 -- 172.21.15.160:0/920006362 learned_addr learned my addr 172.21.15.160:0/920006362 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571b6a640 1 --2- 172.21.15.160:0/920006362 >> v2:172.21.15.160:3300/0 conn(0x7fd56c06cd20 0x7fd56c1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58388/0 (socket says 172.21.15.160:58388) 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571369640 1 --2- 172.21.15.160:0/920006362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c0c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571b6a640 1 -- 172.21.15.160:0/920006362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 msgr2=0x7fd56c0c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571b6a640 1 --2- 172.21.15.160:0/920006362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c0c48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571b6a640 1 -- 172.21.15.160:0/920006362 >> v1:172.21.15.160:6789/0 conn(0x7fd56c0c4df0 legacy=0x7fd56c14ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd570b68640 1 -- 172.21.15.160:0/920006362 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 606854347 0 0) 0x7fd56c0c3090 con 0x7fd56c0c4df0 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.414+0000 7fd571b6a640 1 -- 172.21.15.160:0/920006362 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c0c1b10 con 0x7fd56c06cd20 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd571b6a640 1 --2- 172.21.15.160:0/920006362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c06cd20 0x7fd56c1526a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5680097a0 tx=0x7fd56802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a804d27b0b7411 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd570b68640 1 -- 172.21.15.160:0/920006362 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568037070 con 0x7fd56c06cd20 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd570b68640 1 -- 172.21.15.160:0/920006362 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568032440 con 0x7fd56c06cd20 2024-05-09T07:13:55.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd570b68640 1 -- 172.21.15.160:0/920006362 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568032810 con 0x7fd56c06cd20 2024-05-09T07:13:55.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 -- 172.21.15.160:0/920006362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c06cd20 msgr2=0x7fd56c1526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 --2- 172.21.15.160:0/920006362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c06cd20 0x7fd56c1526a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fd5680097a0 tx=0x7fd56802a180 comp rx=0 tx=0).stop 2024-05-09T07:13:55.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 -- 172.21.15.160:0/920006362 shutdown_connections 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 --2- 172.21.15.160:0/920006362 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c0c48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 --2- 172.21.15.160:0/920006362 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c06cd20 0x7fd56c1526a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 -- 172.21.15.160:0/920006362 >> 172.21.15.160:0/920006362 conn(0x7fd56c0c69b0 msgr2=0x7fd56c0c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 -- 172.21.15.160:0/920006362 shutdown_connections 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.415+0000 7fd573dce640 1 -- 172.21.15.160:0/920006362 wait complete. 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.416+0000 7fd573dce640 1 Processor -- start 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.416+0000 7fd573dce640 1 -- start start 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd573dce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd56c06cd20 0x7fd56c0ab290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd573dce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c13ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd573dce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 0x7fd56c13bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd573dce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c06d280 con 0x7fd56c0c4df0 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd573dce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c06d100 con 0x7fd56c06cd20 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd573dce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd56c0c2030 con 0x7fd56c0c2bd0 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 0x7fd56c13bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 0x7fd56c13bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58392/0 (socket says 172.21.15.160:58392) 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 -- 172.21.15.160:0/148777099 learned_addr learned my addr 172.21.15.160:0/148777099 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd571369640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c13ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd571b6a640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd56c06cd20 0x7fd56c0ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 -- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 msgr2=0x7fd56c13ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c13ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 -- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd56c06cd20 msgr2=0x7fd56c0ab290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.417+0000 7fd57236b640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd56c06cd20 0x7fd56c0ab290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd57236b640 1 -- 172.21.15.160:0/148777099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c072ff0 con 0x7fd56c0c4df0 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd571369640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c13ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd571b6a640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd56c06cd20 0x7fd56c0ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd57236b640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 0x7fd56c13bf80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c6d0 tx=0x7fd56400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd564018070 con 0x7fd56c0c4df0 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd564004d60 con 0x7fd56c0c4df0 2024-05-09T07:13:55.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c13d850 con 0x7fd56c0c4df0 2024-05-09T07:13:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56400a880 con 0x7fd56c0c4df0 2024-05-09T07:13:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.418+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c0a8240 con 0x7fd56c0c4df0 2024-05-09T07:13:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.419+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56c0a8060 con 0x7fd56c0c4df0 2024-05-09T07:13:55.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.419+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd56401b070 con 0x7fd56c0c4df0 2024-05-09T07:13:55.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.420+0000 7fd5627fc640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd55006be30 0x7fd55006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.420+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd56408a8b0 con 0x7fd56c0c4df0 2024-05-09T07:13:55.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.420+0000 7fd571b6a640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd55006be30 0x7fd55006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.420+0000 7fd571b6a640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd55006be30 0x7fd55006e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd568034000 tx=0x7fd56802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.424+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd56c0a8060 con 0x7fd56c0c4df0 2024-05-09T07:13:55.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.601+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd56c075650 con 0x7fd56c0c4df0 2024-05-09T07:13:55.603 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.602+0000 7fd5627fc640 1 -- 172.21.15.160:0/148777099 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fd564058f20 con 0x7fd56c0c4df0 2024-05-09T07:13:55.604 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd55006be30 msgr2=0x7fd55006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd55006be30 0x7fd55006e2f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd568034000 tx=0x7fd56802e000 comp rx=0 tx=0).stop 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 msgr2=0x7fd56c13bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 0x7fd56c13bf80 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd56400c6d0 tx=0x7fd56400cb90 comp rx=0 tx=0).stop 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 shutdown_connections 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd55006be30 0x7fd55006e2f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd56c0c4df0 0x7fd56c13bf80 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd56c0c2bd0 0x7fd56c13ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 --2- 172.21.15.160:0/148777099 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd56c06cd20 0x7fd56c0ab290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 >> 172.21.15.160:0/148777099 conn(0x7fd56c0c69b0 msgr2=0x7fd56c14c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:55.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 shutdown_connections 2024-05-09T07:13:55.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.604+0000 7fd573dce640 1 -- 172.21.15.160:0/148777099 wait complete. 2024-05-09T07:13:55.616 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:13:55.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.717+0000 7fa4bb41c640 1 Processor -- start 2024-05-09T07:13:55.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4bb41c640 1 -- start start 2024-05-09T07:13:55.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4bb41c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b406c3e0 0x7fa4b40c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4bb41c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa4b4066f90 0x7fa4b40c0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4bb41c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa4b40ac3b0 con 0x7fa4b414b990 2024-05-09T07:13:55.721 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4bb41c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa4b4155f10 con 0x7fa4b4066f90 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4bb41c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b40ac650 con 0x7fa4b406c3e0 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.718+0000 7fa4b99b9640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa4b414b990 0x7fa4b40ab3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36360/0 (socket says 172.21.15.160:36360) 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b99b9640 1 -- 172.21.15.160:0/635521265 learned_addr learned my addr 172.21.15.160:0/635521265 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b89b7640 1 --2- 172.21.15.160:0/635521265 >> v2:172.21.15.188:3300/0 conn(0x7fa4b4066f90 0x7fa4b40c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b91b8640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b406c3e0 0x7fa4b40c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4a7fff640 1 -- 172.21.15.160:0/635521265 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1180290071 0 0) 0x7fa4b40ac3b0 con 0x7fa4b414b990 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4a7fff640 1 -- 172.21.15.160:0/635521265 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa4a0003520 con 0x7fa4b414b990 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b89b7640 1 -- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b406c3e0 msgr2=0x7fa4b40c20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b89b7640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b406c3e0 0x7fa4b40c20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b89b7640 1 -- 172.21.15.160:0/635521265 >> v1:172.21.15.160:6789/0 conn(0x7fa4b414b990 legacy=0x7fa4b40ab3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4a7fff640 1 -- 172.21.15.160:0/635521265 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4046384187 0 0) 0x7fa4a0003520 con 0x7fa4b414b990 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b89b7640 1 -- 172.21.15.160:0/635521265 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b41561f0 con 0x7fa4b4066f90 2024-05-09T07:13:55.722 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.719+0000 7fa4b91b8640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b406c3e0 0x7fa4b40c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:13:55.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4b89b7640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00097a0 tx=0x7fa4b002a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6476d39d58a431ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4a7fff640 1 -- 172.21.15.160:0/635521265 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0037070 con 0x7fa4b4066f90 2024-05-09T07:13:55.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4a7fff640 1 -- 172.21.15.160:0/635521265 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b002ab10 con 0x7fa4b4066f90 2024-05-09T07:13:55.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4a7fff640 1 -- 172.21.15.160:0/635521265 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4ac0019f0 con 0x7fa4b4066f90 2024-05-09T07:13:55.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4bb41c640 1 -- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 msgr2=0x7fa4b40c0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0760 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa4b00097a0 tx=0x7fa4b002a080 comp rx=0 tx=0).stop 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4bb41c640 1 -- 172.21.15.160:0/635521265 shutdown_connections 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0760 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/635521265 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b406c3e0 0x7fa4b40c20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.720+0000 7fa4bb41c640 1 -- 172.21.15.160:0/635521265 >> 172.21.15.160:0/635521265 conn(0x7fa4b40c3e30 msgr2=0x7fa4b40c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.721+0000 7fa4bb41c640 1 -- 172.21.15.160:0/635521265 shutdown_connections 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.721+0000 7fa4bb41c640 1 -- 172.21.15.160:0/635521265 wait complete. 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 Processor -- start 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 -- start start 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 0x7fa4b4144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b414b990 0x7fa4b4144780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.726 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b40c4d80 con 0x7fa4b406c3e0 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa4b40c4c00 con 0x7fa4b4066f90 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4bb41c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa4b406c7c0 con 0x7fa4b414b990 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4b89b7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 0x7fa4b4144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4b89b7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 0x7fa4b4144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58404/0 (socket says 172.21.15.160:58404) 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.722+0000 7fa4b89b7640 1 -- 172.21.15.160:0/862584482 learned_addr learned my addr 172.21.15.160:0/862584482 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b91b8640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b99b9640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b414b990 0x7fa4b4144780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b89b7640 1 -- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b414b990 msgr2=0x7fa4b4144780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b89b7640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b414b990 0x7fa4b4144780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b89b7640 1 -- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 msgr2=0x7fa4b40c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b89b7640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b89b7640 1 -- 172.21.15.160:0/862584482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4b41453f0 con 0x7fa4b406c3e0 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b91b8640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4b89b7640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 0x7fa4b4144240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0035040 tx=0x7fa4b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.727 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0041020 con 0x7fa4b406c3e0 2024-05-09T07:13:55.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4b003d620 con 0x7fa4b406c3e0 2024-05-09T07:13:55.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4b41eec60 con 0x7fa4b406c3e0 2024-05-09T07:13:55.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4b0037040 con 0x7fa4b406c3e0 2024-05-09T07:13:55.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.723+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4b41eef40 con 0x7fa4b406c3e0 2024-05-09T07:13:55.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.724+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4b406dd30 con 0x7fa4b406c3e0 2024-05-09T07:13:55.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.725+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa4b0044020 con 0x7fa4b406c3e0 2024-05-09T07:13:55.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.725+0000 7fa4a5ffb640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa49806be30 0x7fa49806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:55.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.725+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4b00ab710 con 0x7fa4b406c3e0 2024-05-09T07:13:55.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.725+0000 7fa4b91b8640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa49806be30 0x7fa49806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:55.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.725+0000 7fa4b91b8640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa49806be30 0x7fa49806e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009a20 tx=0x7fa4a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:55.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.729+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa4b004b030 con 0x7fa4b406c3e0 2024-05-09T07:13:55.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.902+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa4b4075650 con 0x7fa4b406c3e0 2024-05-09T07:13:55.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.902+0000 7fa4a5ffb640 1 -- 172.21.15.160:0/862584482 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fa4b0079d80 con 0x7fa4b406c3e0 2024-05-09T07:13:55.904 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:13:55.906 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa49806be30 msgr2=0x7fa49806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa49806be30 0x7fa49806e2f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa4a8009a20 tx=0x7fa4a8008040 comp rx=0 tx=0).stop 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 msgr2=0x7fa4b4144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 0x7fa4b4144240 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa4b0035040 tx=0x7fa4b00046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 shutdown_connections 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa49806be30 0x7fa49806e2f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa4b414b990 0x7fa4b4144780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa4b406c3e0 0x7fa4b4144240 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 --2- 172.21.15.160:0/862584482 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa4b4066f90 0x7fa4b40c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 >> 172.21.15.160:0/862584482 conn(0x7fa4b40c3e30 msgr2=0x7fa4b40c6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 shutdown_connections 2024-05-09T07:13:55.907 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:55.904+0000 7fa4bb41c640 1 -- 172.21.15.160:0/862584482 wait complete. 2024-05-09T07:13:55.917 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:13:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.017+0000 7f9ab837d640 1 Processor -- start 2024-05-09T07:13:56.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.017+0000 7f9ab837d640 1 -- start start 2024-05-09T07:13:56.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab837d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9ab00a6270 0x7f9ab00c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab837d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00aa1d0 0x7f9ab00aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab837d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ab0155710 con 0x7f9ab00c59c0 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab837d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f9ab00aaba0 con 0x7f9ab00a6270 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab837d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab006dab0 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9ab00a6270 0x7f9ab00c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9ab00a6270 0x7f9ab00c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58412/0 (socket says 172.21.15.160:58412) 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 -- 172.21.15.160:0/1556343501 learned_addr learned my addr 172.21.15.160:0/1556343501 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab691a640 1 --2- 172.21.15.160:0/1556343501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00aa1d0 0x7f9ab00aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 -- 172.21.15.160:0/1556343501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00aa1d0 msgr2=0x7f9ab00aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 --2- 172.21.15.160:0/1556343501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00aa1d0 0x7f9ab00aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 -- 172.21.15.160:0/1556343501 >> v1:172.21.15.160:6789/0 conn(0x7f9ab00c59c0 legacy=0x7f9ab006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5918640 1 -- 172.21.15.160:0/1556343501 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01559b0 con 0x7f9ab00a6270 2024-05-09T07:13:56.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.018+0000 7f9ab5117640 1 -- 172.21.15.160:0/1556343501 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1838938567 0 0) 0x7f9ab0155710 con 0x7f9ab00c59c0 2024-05-09T07:13:56.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.019+0000 7f9ab5918640 1 --2- 172.21.15.160:0/1556343501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00a6270 0x7f9ab00c5da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000bb10 tx=0x7f9aa002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d533f45f38e243 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.019+0000 7f9ab5117640 1 -- 172.21.15.160:0/1556343501 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0036020 con 0x7f9ab00a6270 2024-05-09T07:13:56.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.019+0000 7f9ab5117640 1 -- 172.21.15.160:0/1556343501 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa0032820 con 0x7f9ab00a6270 2024-05-09T07:13:56.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.019+0000 7f9ab5117640 1 -- 172.21.15.160:0/1556343501 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0032bf0 con 0x7f9ab00a6270 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 -- 172.21.15.160:0/1556343501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00a6270 msgr2=0x7f9ab00c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 --2- 172.21.15.160:0/1556343501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00a6270 0x7f9ab00c5da0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000bb10 tx=0x7f9aa002c920 comp rx=0 tx=0).stop 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 -- 172.21.15.160:0/1556343501 shutdown_connections 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 --2- 172.21.15.160:0/1556343501 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00aa1d0 0x7f9ab00aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 --2- 172.21.15.160:0/1556343501 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00a6270 0x7f9ab00c5da0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 -- 172.21.15.160:0/1556343501 >> 172.21.15.160:0/1556343501 conn(0x7f9ab00c69b0 msgr2=0x7f9ab00c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 -- 172.21.15.160:0/1556343501 shutdown_connections 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.020+0000 7f9ab837d640 1 -- 172.21.15.160:0/1556343501 wait complete. 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.021+0000 7f9ab837d640 1 Processor -- start 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.021+0000 7f9ab837d640 1 -- start start 2024-05-09T07:13:56.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00a6270 0x7f9ab01e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 0x7f9ab01dd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ab00c59c0 0x7f9ab0148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab014fa20 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0155f70 con 0x7f9ab00c59c0 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9ab014fba0 con 0x7f9ab00a6270 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 0x7f9ab01dd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 0x7f9ab01dd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58424/0 (socket says 172.21.15.160:58424) 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 -- 172.21.15.160:0/3739250942 learned_addr learned my addr 172.21.15.160:0/3739250942 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab6119640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00a6270 0x7f9ab01e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 -- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00a6270 msgr2=0x7f9ab01e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00a6270 0x7f9ab01e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab691a640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ab00c59c0 0x7f9ab0148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 -- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ab00c59c0 msgr2=0x7f9ab0148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ab00c59c0 0x7f9ab0148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 -- 172.21.15.160:0/3739250942 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab0073210 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab5918640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 0x7f9ab01dd1c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9aa002c960 tx=0x7f9aa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0036040 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9aa003cd70 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0046610 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab00734f0 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.022+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab00a6730 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.023+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab0075650 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.024+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9aa004d060 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.024+0000 7f9aa6ffd640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9a9406be60 0x7f9a9406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.024+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9aa00ae190 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.024+0000 7f9ab6119640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9a9406be60 0x7f9a9406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.025+0000 7f9ab6119640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9a9406be60 0x7f9a9406e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9aac000f60 tx=0x7f9aac009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.029+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9aa0041070 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.202+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ab00c4270 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.202+0000 7f9aa6ffd640 1 -- 172.21.15.160:0/3739250942 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f9aa002f070 con 0x7f9ab00aa1d0 2024-05-09T07:13:56.204 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:13:56.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9a9406be60 msgr2=0x7f9a9406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9a9406be60 0x7f9a9406e320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9aac000f60 tx=0x7f9aac009040 comp rx=0 tx=0).stop 2024-05-09T07:13:56.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 msgr2=0x7f9ab01dd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 0x7f9ab01dd1c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9aa002c960 tx=0x7f9aa00046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 shutdown_connections 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9a9406be60 0x7f9a9406e320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9ab00c59c0 0x7f9ab0148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ab00aa1d0 0x7f9ab01dd1c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 --2- 172.21.15.160:0/3739250942 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9ab00a6270 0x7f9ab01e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 >> 172.21.15.160:0/3739250942 conn(0x7f9ab00c69b0 msgr2=0x7f9ab006d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 shutdown_connections 2024-05-09T07:13:56.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.204+0000 7f9ab837d640 1 -- 172.21.15.160:0/3739250942 wait complete. 2024-05-09T07:13:56.217 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:13:56.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.317+0000 7f6f8c109640 1 Processor -- start 2024-05-09T07:13:56.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.318+0000 7f6f8c109640 1 -- start start 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.318+0000 7f6f8c109640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6f840c48b0 0x7f6f840c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.318+0000 7f6f8c109640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f84153550 con 0x7f6f84150800 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.318+0000 7f6f8c109640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6f84153830 con 0x7f6f840a9a60 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.318+0000 7f6f8c109640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6f84153b10 con 0x7f6f840c48b0 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f89ea5640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6f84150800 0x7f6f840a5e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36384/0 (socket says 172.21.15.160:36384) 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f89ea5640 1 -- 172.21.15.160:0/1175163392 learned_addr learned my addr 172.21.15.160:0/1175163392 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f8a6a6640 1 --2- 172.21.15.160:0/1175163392 >> v2:172.21.15.188:3300/0 conn(0x7f6f840c48b0 0x7f6f840c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 78086373 0 0) 0x7f6f84153550 con 0x7f6f84150800 2024-05-09T07:13:56.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f70003520 con 0x7f6f84150800 2024-05-09T07:13:56.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1674957882 0 0) 0x7f6f84153830 con 0x7f6f840a9a60 2024-05-09T07:13:56.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6f84153550 con 0x7f6f840a9a60 2024-05-09T07:13:56.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/1175163392 >> v1:172.21.15.188:6789/0 conn(0x7f6f840a9a60 legacy=0x7f6f840c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/1175163392 >> v1:172.21.15.160:6789/0 conn(0x7f6f84150800 legacy=0x7f6f840a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.319+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/1175163392 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f84153df0 con 0x7f6f840c48b0 2024-05-09T07:13:56.326 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f8a6a6640 1 --2- 172.21.15.160:0/1175163392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840c48b0 0x7f6f840c4d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009d90 tx=0x7f6f7c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6adb3e699451bcd9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c007740 con 0x7f6f840c48b0 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c0078e0 con 0x7f6f840c48b0 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f88ea3640 1 -- 172.21.15.160:0/1175163392 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c007cb0 con 0x7f6f840c48b0 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f8c109640 1 -- 172.21.15.160:0/1175163392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840c48b0 msgr2=0x7f6f840c4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f8c109640 1 --2- 172.21.15.160:0/1175163392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840c48b0 0x7f6f840c4d20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009d90 tx=0x7f6f7c02a660 comp rx=0 tx=0).stop 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f8c109640 1 -- 172.21.15.160:0/1175163392 shutdown_connections 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f8c109640 1 --2- 172.21.15.160:0/1175163392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840c48b0 0x7f6f840c4d20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.320+0000 7f6f8c109640 1 -- 172.21.15.160:0/1175163392 >> 172.21.15.160:0/1175163392 conn(0x7f6f840c69b0 msgr2=0x7f6f840c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.321+0000 7f6f8c109640 1 -- 172.21.15.160:0/1175163392 shutdown_connections 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.321+0000 7f6f8c109640 1 -- 172.21.15.160:0/1175163392 wait complete. 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 Processor -- start 2024-05-09T07:13:56.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 -- start start 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840a9a60 0x7f6f84154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f840c48b0 0x7f6f840a9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 0x7f6f841e6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6f840c65e0 con 0x7f6f84150800 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6f84153200 con 0x7f6f840a9a60 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8c109640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6f840c6760 con 0x7f6f840c48b0 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8a6a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 0x7f6f841e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8a6a6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 0x7f6f841e6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58434/0 (socket says 172.21.15.160:58434) 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.322+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/4102415760 learned_addr learned my addr 172.21.15.160:0/4102415760 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f89ea5640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840a9a60 0x7f6f84154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f896a4640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f840c48b0 0x7f6f840a9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f840c48b0 msgr2=0x7f6f840a9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8a6a6640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f840c48b0 0x7f6f840a9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840a9a60 msgr2=0x7f6f84154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8a6a6640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840a9a60 0x7f6f84154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8a6a6640 1 -- 172.21.15.160:0/4102415760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f841e6690 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8a6a6640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 0x7f6f841e6150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009eb0 tx=0x7f6f7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c040070 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6f7c031bc0 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f840a8750 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c009450 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.323+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f84157230 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.324+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f84075650 con 0x7f6f84150800 2024-05-09T07:13:56.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.325+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6f7c039040 con 0x7f6f84150800 2024-05-09T07:13:56.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.325+0000 7f6f767fc640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6f6806be30 0x7f6f6806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.325+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6f7c0ac740 con 0x7f6f84150800 2024-05-09T07:13:56.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.325+0000 7f6f89ea5640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6f6806be30 0x7f6f6806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.326+0000 7f6f89ea5640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6f6806be30 0x7f6f6806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f78000f60 tx=0x7f6f78007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.329+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6f841e6690 con 0x7f6f84150800 2024-05-09T07:13:56.507 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.504+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6f8414f060 con 0x7f6f84150800 2024-05-09T07:13:56.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.506+0000 7f6f767fc640 1 -- 172.21.15.160:0/4102415760 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f6f7c07a7f0 con 0x7f6f84150800 2024-05-09T07:13:56.508 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6f6806be30 msgr2=0x7f6f6806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6f6806be30 0x7f6f6806e2f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f78000f60 tx=0x7f6f78007d50 comp rx=0 tx=0).stop 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 msgr2=0x7f6f841e6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 0x7f6f841e6150 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009eb0 tx=0x7f6f7c004a40 comp rx=0 tx=0).stop 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 shutdown_connections 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6f6806be30 0x7f6f6806e2f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6f84150800 0x7f6f841e6150 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6f840c48b0 0x7f6f840a9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 --2- 172.21.15.160:0/4102415760 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6f840a9a60 0x7f6f84154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.508+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 >> 172.21.15.160:0/4102415760 conn(0x7f6f840c69b0 msgr2=0x7f6f8414aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.509+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 shutdown_connections 2024-05-09T07:13:56.512 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.509+0000 7f6f8c109640 1 -- 172.21.15.160:0/4102415760 wait complete. 2024-05-09T07:13:56.522 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T07:13:56.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.621+0000 7efef456a640 1 Processor -- start 2024-05-09T07:13:56.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.621+0000 7efef456a640 1 -- start start 2024-05-09T07:13:56.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef456a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0a5fb0 0x7efeec0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef456a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeec1563b0 con 0x7efeec0bc360 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef456a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeec156690 con 0x7efeec153300 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef456a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efeec0c1630 con 0x7efeec0a5fb0 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef2b07640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7efeec0bc360 0x7efeec15cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36392/0 (socket says 172.21.15.160:36392) 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef2b07640 1 -- 172.21.15.160:0/228695386 learned_addr learned my addr 172.21.15.160:0/228695386 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.622+0000 7efef1b05640 1 --2- 172.21.15.160:0/228695386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0a5fb0 0x7efeec0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1848354939 0 0) 0x7efeec1563b0 con 0x7efeec0bc360 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efed8003520 con 0x7efeec0bc360 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 187862890 0 0) 0x7efeec156690 con 0x7efeec153300 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7efeec1563b0 con 0x7efeec153300 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1b05640 1 -- 172.21.15.160:0/228695386 >> v1:172.21.15.188:6789/0 conn(0x7efeec153300 legacy=0x7efeec0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1b05640 1 -- 172.21.15.160:0/228695386 >> v1:172.21.15.160:6789/0 conn(0x7efeec0bc360 legacy=0x7efeec15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1b05640 1 -- 172.21.15.160:0/228695386 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec0c1c20 con 0x7efeec0a5fb0 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.623+0000 7efef1b05640 1 --2- 172.21.15.160:0/228695386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0a5fb0 0x7efeec0bbe20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efedc0097d0 tx=0x7efedc02c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dfb0dd102a54800 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc038070 con 0x7efeec0a5fb0 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efedc031710 con 0x7efeec0a5fb0 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef1304640 1 -- 172.21.15.160:0/228695386 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc031ac0 con 0x7efeec0a5fb0 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 -- 172.21.15.160:0/228695386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0a5fb0 msgr2=0x7efeec0bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 --2- 172.21.15.160:0/228695386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0a5fb0 0x7efeec0bbe20 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7efedc0097d0 tx=0x7efedc02c420 comp rx=0 tx=0).stop 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 -- 172.21.15.160:0/228695386 shutdown_connections 2024-05-09T07:13:56.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 --2- 172.21.15.160:0/228695386 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0a5fb0 0x7efeec0bbe20 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 -- 172.21.15.160:0/228695386 >> 172.21.15.160:0/228695386 conn(0x7efeec0c7580 msgr2=0x7efeec0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 -- 172.21.15.160:0/228695386 shutdown_connections 2024-05-09T07:13:56.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.624+0000 7efef456a640 1 -- 172.21.15.160:0/228695386 wait complete. 2024-05-09T07:13:56.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.625+0000 7efef456a640 1 Processor -- start 2024-05-09T07:13:56.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.625+0000 7efef456a640 1 -- start start 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef456a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeec0a5fb0 0x7efeec1ea7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef456a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0bc360 0x7efeec152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef456a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 0x7efeec0aab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef456a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efeec0c2420 con 0x7efeec153300 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef456a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efeec0c19d0 con 0x7efeec0a5fb0 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef456a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efeec0bc7d0 con 0x7efeec0bc360 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2b07640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 0x7efeec0aab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2b07640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 0x7efeec0aab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58446/0 (socket says 172.21.15.160:58446) 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2b07640 1 -- 172.21.15.160:0/573365836 learned_addr learned my addr 172.21.15.160:0/573365836 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef1b05640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0bc360 0x7efeec152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2b07640 1 -- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0bc360 msgr2=0x7efeec152380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2b07640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0bc360 0x7efeec152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2b07640 1 -- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeec0a5fb0 msgr2=0x7efeec1ea7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.626+0000 7efef2306640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeec0a5fb0 0x7efeec1ea7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efef2b07640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeec0a5fb0 0x7efeec1ea7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efef2b07640 1 -- 172.21.15.160:0/573365836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeec0beba0 con 0x7efeec153300 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efef2b07640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 0x7efeec0aab40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efee400cb40 tx=0x7efee400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4017070 con 0x7efeec153300 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee400aca0 con 0x7efeec153300 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeec154830 con 0x7efeec153300 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee4012600 con 0x7efeec153300 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.627+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeec154df0 con 0x7efeec153300 2024-05-09T07:13:56.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.628+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeec0756c0 con 0x7efeec153300 2024-05-09T07:13:56.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.629+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efee401a070 con 0x7efeec153300 2024-05-09T07:13:56.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.629+0000 7efee2ffd640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efed006be30 0x7efed006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.629+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7efee408ae90 con 0x7efeec153300 2024-05-09T07:13:56.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.629+0000 7efef2306640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efed006be30 0x7efed006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.629+0000 7efef2306640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efed006be30 0x7efed006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efee8007fd0 tx=0x7efee8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.633+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efee4058e30 con 0x7efeec153300 2024-05-09T07:13:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.820+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efeec0bdab0 con 0x7efeec153300 2024-05-09T07:13:56.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.821+0000 7efee2ffd640 1 -- 172.21.15.160:0/573365836 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7efee405d130 con 0x7efeec153300 2024-05-09T07:13:56.823 INFO:teuthology.orchestra.run.smithi160.stderr:listed 0 entries 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efed006be30 msgr2=0x7efed006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efed006be30 0x7efed006e2f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efee8007fd0 tx=0x7efee8009040 comp rx=0 tx=0).stop 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 msgr2=0x7efeec0aab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 0x7efeec0aab40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7efee400cb40 tx=0x7efee400a060 comp rx=0 tx=0).stop 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 shutdown_connections 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efed006be30 0x7efed006e2f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeec153300 0x7efeec0aab40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeec0bc360 0x7efeec152380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 --2- 172.21.15.160:0/573365836 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeec0a5fb0 0x7efeec1ea7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 >> 172.21.15.160:0/573365836 conn(0x7efeec0c7580 msgr2=0x7efeec0c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 shutdown_connections 2024-05-09T07:13:56.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.823+0000 7efef456a640 1 -- 172.21.15.160:0/573365836 wait complete. 2024-05-09T07:13:56.835 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:13:56.835 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:13:56.852 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:56.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.991+0000 7fc74893b640 1 Processor -- start 2024-05-09T07:13:56.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.991+0000 7fc74893b640 1 -- start start 2024-05-09T07:13:56.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.991+0000 7fc74893b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.994 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74893b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc7440bc360 0x7fc74415cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74893b640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc7441563b0 con 0x7fc744153300 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74893b640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc7440c1630 con 0x7fc7440bc360 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74893b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc7440c19d0 con 0x7fc7440a5fb0 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc742d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc7440bc360 0x7fc74415cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74259c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc744153300 0x7fc7440c58e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36400/0 (socket says 172.21.15.160:36400) 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74259c640 1 -- 172.21.15.160:0/3752221864 learned_addr learned my addr 172.21.15.160:0/3752221864 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc741d9b640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc742d9d640 1 -- 172.21.15.160:0/3752221864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 msgr2=0x7fc7440bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc742d9d640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc742d9d640 1 -- 172.21.15.160:0/3752221864 >> v1:172.21.15.160:6789/0 conn(0x7fc744153300 legacy=0x7fc7440c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc742d9d640 1 -- 172.21.15.160:0/3752221864 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc744156690 con 0x7fc7440bc360 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc74159a640 1 -- 172.21.15.160:0/3752221864 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 362336631 0 0) 0x7fc7441563b0 con 0x7fc744153300 2024-05-09T07:13:56.995 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.992+0000 7fc741d9b640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:56.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc742d9d640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7440bc360 0x7fc74415cc30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7340097a0 tx=0x7fc73402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771774896d6bcea6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:56.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74159a640 1 -- 172.21.15.160:0/3752221864 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734037070 con 0x7fc7440bc360 2024-05-09T07:13:56.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74159a640 1 -- 172.21.15.160:0/3752221864 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc734032440 con 0x7fc7440bc360 2024-05-09T07:13:56.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74159a640 1 -- 172.21.15.160:0/3752221864 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734032810 con 0x7fc7440bc360 2024-05-09T07:13:56.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74893b640 1 -- 172.21.15.160:0/3752221864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7440bc360 msgr2=0x7fc74415cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:56.998 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74893b640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7440bc360 0x7fc74415cc30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7340097a0 tx=0x7fc73402a180 comp rx=0 tx=0).stop 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74893b640 1 -- 172.21.15.160:0/3752221864 shutdown_connections 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74893b640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc7440bc360 0x7fc74415cc30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74893b640 1 --2- 172.21.15.160:0/3752221864 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440bbe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.993+0000 7fc74893b640 1 -- 172.21.15.160:0/3752221864 >> 172.21.15.160:0/3752221864 conn(0x7fc7440c7580 msgr2=0x7fc7440a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.994+0000 7fc74893b640 1 -- 172.21.15.160:0/3752221864 shutdown_connections 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.994+0000 7fc74893b640 1 -- 172.21.15.160:0/3752221864 wait complete. 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.995+0000 7fc74893b640 1 Processor -- start 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.995+0000 7fc74893b640 1 -- start start 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.995+0000 7fc74893b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.995+0000 7fc74893b640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7440bc360 0x7fc74414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.995+0000 7fc74893b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 0x7fc744150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc74893b640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc7440c2420 con 0x7fc744153300 2024-05-09T07:13:56.999 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc74893b640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc7440c1f00 con 0x7fc7440bc360 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc74893b640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc7440bc7d0 con 0x7fc7440a5fb0 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 0x7fc744150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 0x7fc744150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58462/0 (socket says 172.21.15.160:58462) 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 -- 172.21.15.160:0/489953078 learned_addr learned my addr 172.21.15.160:0/489953078 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc74259c640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc741d9b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7440bc360 0x7fc74414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 -- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 msgr2=0x7fc7440aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 -- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7440bc360 msgr2=0x7fc74414fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7440bc360 0x7fc74414fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 -- 172.21.15.160:0/489953078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc744070c70 con 0x7fc744153300 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc742d9d640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 0x7fc744150150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc734035040 tx=0x7fc7340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734037040 con 0x7fc744153300 2024-05-09T07:13:57.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc734042070 con 0x7fc744153300 2024-05-09T07:13:57.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc734030b50 con 0x7fc744153300 2024-05-09T07:13:57.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7441de2a0 con 0x7fc744153300 2024-05-09T07:13:57.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.996+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc744073550 con 0x7fc744153300 2024-05-09T07:13:57.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.997+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7440756c0 con 0x7fc744153300 2024-05-09T07:13:57.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.998+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc734030cf0 con 0x7fc744153300 2024-05-09T07:13:57.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.998+0000 7fc732ffd640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc72406be30 0x7fc72406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.998+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc7340abae0 con 0x7fc744153300 2024-05-09T07:13:57.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.998+0000 7fc74259c640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc72406be30 0x7fc72406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.002 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:56.999+0000 7fc74259c640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc72406be30 0x7fc72406e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000f60 tx=0x7fc73c007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.004 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.002+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc734032e20 con 0x7fc744153300 2024-05-09T07:13:57.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.181+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc7440bdab0 con 0x7fc744153300 2024-05-09T07:13:57.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.182+0000 7fc732ffd640 1 -- 172.21.15.160:0/489953078 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7fc734079b90 con 0x7fc744153300 2024-05-09T07:13:57.184 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:57.184 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc72406be30 msgr2=0x7fc72406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc72406be30 0x7fc72406e2f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc73c000f60 tx=0x7fc73c007d50 comp rx=0 tx=0).stop 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 msgr2=0x7fc744150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 0x7fc744150150 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc734035040 tx=0x7fc7340046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 shutdown_connections 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc72406be30 0x7fc72406e2f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc7440bc360 0x7fc74414fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc744153300 0x7fc744150150 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 --2- 172.21.15.160:0/489953078 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc7440a5fb0 0x7fc7440aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 >> 172.21.15.160:0/489953078 conn(0x7fc7440c7580 msgr2=0x7fc7440a8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:57.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 shutdown_connections 2024-05-09T07:13:57.187 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.184+0000 7fc74893b640 1 -- 172.21.15.160:0/489953078 wait complete. 2024-05-09T07:13:57.197 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:13:57.197 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:13:57.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.298+0000 7faed0c64640 1 Processor -- start 2024-05-09T07:13:57.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.298+0000 7faed0c64640 1 -- start start 2024-05-09T07:13:57.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faed0c64640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faecc0c59c0 0x7faecc0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faed0c64640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7faecc0aa1d0 0x7faecc0aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faed0c64640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7faecc155710 con 0x7faecc0a6270 2024-05-09T07:13:57.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faed0c64640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7faecc0aaba0 con 0x7faecc0c59c0 2024-05-09T07:13:57.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faed0c64640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7faecc06dab0 con 0x7faecc0aa1d0 2024-05-09T07:13:57.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faeca59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faecc0c59c0 0x7faecc0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faeca59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7faecc0c59c0 0x7faecc0c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58478/0 (socket says 172.21.15.160:58478) 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faeca59c640 1 -- 172.21.15.160:0/641339809 learned_addr learned my addr 172.21.15.160:0/641339809 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.299+0000 7faecad9d640 1 --2- 172.21.15.160:0/641339809 >> v2:172.21.15.188:3300/0 conn(0x7faecc0aa1d0 0x7faecc0aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faeca59c640 1 -- 172.21.15.160:0/641339809 >> v2:172.21.15.188:3300/0 conn(0x7faecc0aa1d0 msgr2=0x7faecc0aa660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faeca59c640 1 --2- 172.21.15.160:0/641339809 >> v2:172.21.15.188:3300/0 conn(0x7faecc0aa1d0 0x7faecc0aa660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faeca59c640 1 -- 172.21.15.160:0/641339809 >> v1:172.21.15.188:6789/0 conn(0x7faecc0a6270 legacy=0x7faecc06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faeca59c640 1 -- 172.21.15.160:0/641339809 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1559b0 con 0x7faecc0c59c0 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faeca59c640 1 --2- 172.21.15.160:0/641339809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0c59c0 0x7faecc0c5da0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faecc0ac130 tx=0x7faebc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31510d31bb2f1c57 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faec9d9b640 1 -- 172.21.15.160:0/641339809 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc037070 con 0x7faecc0c59c0 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faec9d9b640 1 -- 172.21.15.160:0/641339809 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faebc032440 con 0x7faecc0c59c0 2024-05-09T07:13:57.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.300+0000 7faec9d9b640 1 -- 172.21.15.160:0/641339809 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc032810 con 0x7faecc0c59c0 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 -- 172.21.15.160:0/641339809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0c59c0 msgr2=0x7faecc0c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 --2- 172.21.15.160:0/641339809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0c59c0 0x7faecc0c5da0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7faecc0ac130 tx=0x7faebc02a270 comp rx=0 tx=0).stop 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 -- 172.21.15.160:0/641339809 shutdown_connections 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 --2- 172.21.15.160:0/641339809 >> v2:172.21.15.188:3300/0 conn(0x7faecc0aa1d0 0x7faecc0aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 --2- 172.21.15.160:0/641339809 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0c59c0 0x7faecc0c5da0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 -- 172.21.15.160:0/641339809 >> 172.21.15.160:0/641339809 conn(0x7faecc0c69b0 msgr2=0x7faecc0c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 -- 172.21.15.160:0/641339809 shutdown_connections 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.301+0000 7faed0c64640 1 -- 172.21.15.160:0/641339809 wait complete. 2024-05-09T07:13:57.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.302+0000 7faed0c64640 1 Processor -- start 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 -- start start 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 0x7faecc0a6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faecc0aa1d0 0x7faecc1e5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faecc0c59c0 0x7faecc13e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7faecc14fa20 con 0x7faecc0a6270 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7faecc155f70 con 0x7faecc0c59c0 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faed0c64640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7faecc14fba0 con 0x7faecc0aa1d0 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faeca59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 0x7faecc0a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faeca59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 0x7faecc0a6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58484/0 (socket says 172.21.15.160:58484) 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faeca59c640 1 -- 172.21.15.160:0/2706561149 learned_addr learned my addr 172.21.15.160:0/2706561149 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faecad9d640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faecc0c59c0 0x7faecc13e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.303+0000 7faec1d9b640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faecc0aa1d0 0x7faecc1e5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faeca59c640 1 -- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faecc0aa1d0 msgr2=0x7faecc1e5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faeca59c640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faecc0aa1d0 0x7faecc1e5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faeca59c640 1 -- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faecc0c59c0 msgr2=0x7faecc13e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faeca59c640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faecc0c59c0 0x7faecc13e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faeca59c640 1 -- 172.21.15.160:0/2706561149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faecc1eb3b0 con 0x7faecc0a6270 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faeca59c640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 0x7faecc0a6670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc041020 con 0x7faecc0a6270 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7faebc0306c0 con 0x7faecc0a6270 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faecc1e4b70 con 0x7faecc0a6270 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc037040 con 0x7faecc0a6270 2024-05-09T07:13:57.308 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.304+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faecc1e58b0 con 0x7faecc0a6270 2024-05-09T07:13:57.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.305+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faecc075650 con 0x7faecc0a6270 2024-05-09T07:13:57.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.305+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7faebc044020 con 0x7faecc0a6270 2024-05-09T07:13:57.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.306+0000 7faec37fe640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7faeac06be30 0x7faeac06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.306+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7faebc0ab7b0 con 0x7faecc0a6270 2024-05-09T07:13:57.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.306+0000 7faec1d9b640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7faeac06be30 0x7faeac06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.309 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.306+0000 7faec1d9b640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7faeac06be30 0x7faeac06e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faecc0a91c0 tx=0x7faeb4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.310+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7faebc04b030 con 0x7faecc0a6270 2024-05-09T07:13:57.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.487+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faecc1eb580 con 0x7faecc0a6270 2024-05-09T07:13:57.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.488+0000 7faec37fe640 1 -- 172.21.15.160:0/2706561149 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7faebc080080 con 0x7faecc0a6270 2024-05-09T07:13:57.489 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7faeac06be30 msgr2=0x7faeac06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7faeac06be30 0x7faeac06e2f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faecc0a91c0 tx=0x7faeb4009040 comp rx=0 tx=0).stop 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 msgr2=0x7faecc0a6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 0x7faecc0a6670 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faebc000c00 tx=0x7faebc0046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 shutdown_connections 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7faeac06be30 0x7faeac06e2f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7faecc0c59c0 0x7faecc13e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7faecc0aa1d0 0x7faecc1e5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 --2- 172.21.15.160:0/2706561149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faecc0a6270 0x7faecc0a6670 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 >> 172.21.15.160:0/2706561149 conn(0x7faecc0c69b0 msgr2=0x7faecc06d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 shutdown_connections 2024-05-09T07:13:57.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.490+0000 7faed0c64640 1 -- 172.21.15.160:0/2706561149 wait complete. 2024-05-09T07:13:57.503 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:13:57.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.606+0000 7f9f998ba640 1 Processor -- start 2024-05-09T07:13:57.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.606+0000 7f9f998ba640 1 -- start start 2024-05-09T07:13:57.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f998ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f94152a80 0x7f9f941568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f998ba640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9f94155e70 0x7f9f94159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f998ba640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9f940c5690 0x7f9f940c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f998ba640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f9f94152e60 con 0x7f9f940c5690 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f998ba640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9f940c80a0 con 0x7f9f94155e70 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f998ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f940c7ae0 con 0x7f9f94152a80 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9f940c5690 0x7f9f940c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9f940c5690 0x7f9f940c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58500/0 (socket says 172.21.15.160:58500) 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 -- 172.21.15.160:0/1936779220 learned_addr learned my addr 172.21.15.160:0/1936779220 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f8a7fc640 1 --2- 172.21.15.160:0/1936779220 >> v2:172.21.15.188:3300/0 conn(0x7f9f94155e70 0x7f9f94159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f92ffd640 1 --2- 172.21.15.160:0/1936779220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f94152a80 0x7f9f941568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 -- 172.21.15.160:0/1936779220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f94152a80 msgr2=0x7f9f941568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 --2- 172.21.15.160:0/1936779220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f94152a80 0x7f9f941568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 -- 172.21.15.160:0/1936779220 >> v2:172.21.15.188:3300/0 conn(0x7f9f94155e70 msgr2=0x7f9f94159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 --2- 172.21.15.160:0/1936779220 >> v2:172.21.15.188:3300/0 conn(0x7f9f94155e70 0x7f9f94159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.607+0000 7f9f937fe640 1 -- 172.21.15.160:0/1936779220 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f940c7dc0 con 0x7f9f940c5690 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.608+0000 7f9f937fe640 1 --2- 172.21.15.160:0/1936779220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f940c5690 0x7f9f940c5ac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f9408c910 tx=0x7f9f8c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a603449aa90f9ba3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.608+0000 7f9f927fc640 1 -- 172.21.15.160:0/1936779220 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c036070 con 0x7f9f940c5690 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.608+0000 7f9f927fc640 1 -- 172.21.15.160:0/1936779220 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f8c031600 con 0x7f9f940c5690 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.608+0000 7f9f927fc640 1 -- 172.21.15.160:0/1936779220 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0319d0 con 0x7f9f940c5690 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 -- 172.21.15.160:0/1936779220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f940c5690 msgr2=0x7f9f940c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 --2- 172.21.15.160:0/1936779220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f940c5690 0x7f9f940c5ac0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9f9408c910 tx=0x7f9f8c02a570 comp rx=0 tx=0).stop 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 -- 172.21.15.160:0/1936779220 shutdown_connections 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 --2- 172.21.15.160:0/1936779220 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f940c5690 0x7f9f940c5ac0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 --2- 172.21.15.160:0/1936779220 >> v2:172.21.15.188:3300/0 conn(0x7f9f94155e70 0x7f9f94159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 --2- 172.21.15.160:0/1936779220 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f94152a80 0x7f9f941568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 -- 172.21.15.160:0/1936779220 >> 172.21.15.160:0/1936779220 conn(0x7f9f94072ec0 msgr2=0x7f9f940ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:57.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 -- 172.21.15.160:0/1936779220 shutdown_connections 2024-05-09T07:13:57.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.609+0000 7f9f998ba640 1 -- 172.21.15.160:0/1936779220 wait complete. 2024-05-09T07:13:57.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.610+0000 7f9f998ba640 1 Processor -- start 2024-05-09T07:13:57.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.610+0000 7f9f998ba640 1 -- start start 2024-05-09T07:13:57.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f998ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f940c5690 0x7f9f94151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f998ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 0x7f9f940cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f998ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f94155e70 0x7f9f940bfba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f998ba640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9f940ad6a0 con 0x7f9f94152a80 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f998ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94158df0 con 0x7f9f94155e70 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f998ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9f941562f0 con 0x7f9f940c5690 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 0x7f9f940cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 0x7f9f940cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58516/0 (socket says 172.21.15.160:58516) 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 -- 172.21.15.160:0/460990246 learned_addr learned my addr 172.21.15.160:0/460990246 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f92ffd640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f940c5690 0x7f9f94151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f937fe640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f94155e70 0x7f9f940bfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 -- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f940c5690 msgr2=0x7f9f94151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f940c5690 0x7f9f94151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 -- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f94155e70 msgr2=0x7f9f940bfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f94155e70 0x7f9f940bfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 -- 172.21.15.160:0/460990246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f941eabc0 con 0x7f9f94152a80 2024-05-09T07:13:57.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f8a7fc640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 0x7f9f940cb3a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00d620 tx=0x7f9f7c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.611+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00b620 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.612+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f7c019070 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.612+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f941eaea0 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.612+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00ba00 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.612+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f941eb460 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.612+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f940ad3a0 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.613+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9f7c00bba0 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.613+0000 7f9f90ff9640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f7406be60 0x7f9f7406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.613+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9f7c08b4e0 con 0x7f9f94152a80 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.613+0000 7f9f92ffd640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f7406be60 0x7f9f7406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.614+0000 7f9f92ffd640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f7406be60 0x7f9f7406e320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f84006fd0 tx=0x7f9f84008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.618+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9f7c059be0 con 0x7f9f94152a80 2024-05-09T07:13:57.793 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.792+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9f94075630 con 0x7f9f94152a80 2024-05-09T07:13:57.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.792+0000 7f9f90ff9640 1 -- 172.21.15.160:0/460990246 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7f9f7c0594e0 con 0x7f9f94152a80 2024-05-09T07:13:57.794 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:13:57.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f7406be60 msgr2=0x7f9f7406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f7406be60 0x7f9f7406e320 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9f84006fd0 tx=0x7f9f84008040 comp rx=0 tx=0).stop 2024-05-09T07:13:57.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 msgr2=0x7f9f940cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.796 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 0x7f9f940cb3a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9f7c00d620 tx=0x7f9f7c00dae0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 shutdown_connections 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9f7406be60 0x7f9f7406e320 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9f94155e70 0x7f9f940bfba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9f94152a80 0x7f9f940cb3a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 --2- 172.21.15.160:0/460990246 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9f940c5690 0x7f9f94151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 >> 172.21.15.160:0/460990246 conn(0x7f9f94072ec0 msgr2=0x7f9f9414af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 shutdown_connections 2024-05-09T07:13:57.797 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.794+0000 7f9f998ba640 1 -- 172.21.15.160:0/460990246 wait complete. 2024-05-09T07:13:57.807 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:13:57.909 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.907+0000 7f859bd12640 1 Processor -- start 2024-05-09T07:13:57.909 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.908+0000 7f859bd12640 1 -- start start 2024-05-09T07:13:57.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.908+0000 7f859bd12640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85940c5690 0x7f85940c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.908+0000 7f859bd12640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8594156d90 con 0x7f85941568e0 2024-05-09T07:13:57.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.908+0000 7f859bd12640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85940c7ae0 con 0x7f8594152a80 2024-05-09T07:13:57.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.908+0000 7f859bd12640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8594152e60 con 0x7f85940c5690 2024-05-09T07:13:57.913 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f85992ad640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f85941568e0 0x7f8594159310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36404/0 (socket says 172.21.15.160:36404) 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f85992ad640 1 -- 172.21.15.160:0/139897014 learned_addr learned my addr 172.21.15.160:0/139897014 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f859a2af640 1 --2- 172.21.15.160:0/139897014 >> v2:172.21.15.188:3300/0 conn(0x7f85940c5690 0x7f85940c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2684989257 0 0) 0x7f8594156d90 con 0x7f85941568e0 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8580003520 con 0x7f85941568e0 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2326184113 0 0) 0x7f85940c7ae0 con 0x7f8594152a80 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8594156d90 con 0x7f8594152a80 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3716977727 0 0) 0x7f8580003520 con 0x7f85941568e0 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 >> v2:172.21.15.188:3300/0 conn(0x7f85940c5690 msgr2=0x7f85940c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 --2- 172.21.15.160:0/139897014 >> v2:172.21.15.188:3300/0 conn(0x7f85940c5690 0x7f85940c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 >> v1:172.21.15.188:6789/0 conn(0x7f8594152a80 legacy=0x7f8594155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85940c7dc0 con 0x7f85941568e0 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f85840015f0 con 0x7f85941568e0 2024-05-09T07:13:57.914 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 >> v1:172.21.15.160:6789/0 conn(0x7f85941568e0 legacy=0x7f8594159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8580003b90 0x7f8580003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f8580004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8580005280 0x7f858000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8594152e60 con 0x7f8580004800 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f858000c1c0 con 0x7f8580003b90 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f858000c3c0 con 0x7f8580005280 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f85992ad640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f8580004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f8599aae640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8580003b90 0x7f8580003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.909+0000 7f859a2af640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8580005280 0x7f858000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f85992ad640 1 -- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8580005280 msgr2=0x7f858000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f85992ad640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8580005280 0x7f858000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f85992ad640 1 -- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8580003b90 msgr2=0x7f8580003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f85992ad640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8580003b90 0x7f8580003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f859a2af640 1 -- 172.21.15.160:0/139897014 reap_dead start 2024-05-09T07:13:57.915 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f85992ad640 1 -- 172.21.15.160:0/139897014 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8580003520 con 0x7f8580004800 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f85992ad640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f8580004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8584007f20 tx=0x7f8580005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d6b3e7b02b87d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584007b40 con 0x7f8580004800 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8584005af0 con 0x7f8580004800 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.910+0000 7f8598aac640 1 -- 172.21.15.160:0/139897014 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8584007070 con 0x7f8580004800 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 -- 172.21.15.160:0/139897014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 msgr2=0x7f8580004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f8580004c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8584007f20 tx=0x7f8580005830 comp rx=0 tx=0).stop 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 -- 172.21.15.160:0/139897014 shutdown_connections 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 --2- 172.21.15.160:0/139897014 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f8580004c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 -- 172.21.15.160:0/139897014 >> 172.21.15.160:0/139897014 conn(0x7f8594072ec0 msgr2=0x7f85940ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:57.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 -- 172.21.15.160:0/139897014 shutdown_connections 2024-05-09T07:13:57.917 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.911+0000 7f859bd12640 1 -- 172.21.15.160:0/139897014 wait complete. 2024-05-09T07:13:57.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.912+0000 7f859bd12640 1 Processor -- start 2024-05-09T07:13:57.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.912+0000 7f859bd12640 1 -- start start 2024-05-09T07:13:57.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859bd12640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f85940be190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859bd12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8594149cf0 0x7f859414a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859bd12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85940af700 0x7f85940afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859bd12640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8594158df0 con 0x7f8580004800 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859bd12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85940c80a0 con 0x7f8594149cf0 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859bd12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f85940ad6a0 con 0x7f85940af700 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f85940be190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f85940be190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58536/0 (socket says 172.21.15.160:58536) 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 -- 172.21.15.160:0/1105361138 learned_addr learned my addr 172.21.15.160:0/1105361138 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f85992ad640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8594149cf0 0x7f859414a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f859a2af640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85940af700 0x7f85940afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 -- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85940af700 msgr2=0x7f85940afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85940af700 0x7f85940afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 -- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8594149cf0 msgr2=0x7f859414a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:57.920 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8594149cf0 0x7f859414a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 -- 172.21.15.160:0/1105361138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85941ebe10 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f8599aae640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f85940be190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f859000f7a0 tx=0x7f859000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.913+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85900050d0 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.914+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8590018070 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.914+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85941ec080 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.914+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859000d650 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.914+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85940b00c0 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.915+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8594075650 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.915+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f859000d7f0 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.916+0000 7f858a7fc640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f857806be60 0x7f857806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.916+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f859008af00 con 0x7f8580004800 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.916+0000 7f85992ad640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f857806be60 0x7f857806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:57.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.916+0000 7f85992ad640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f857806be60 0x7f857806e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8584032000 tx=0x7f8584009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:57.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:57.920+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8590059600 con 0x7f8580004800 2024-05-09T07:13:58.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.097+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f85940ad3a0 con 0x7f8580004800 2024-05-09T07:13:58.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.097+0000 7f858a7fc640 1 -- 172.21.15.160:0/1105361138 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f8590058f00 con 0x7f8580004800 2024-05-09T07:13:58.099 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f857806be60 msgr2=0x7f857806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f857806be60 0x7f857806e320 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8584032000 tx=0x7f8584009030 comp rx=0 tx=0).stop 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 msgr2=0x7f85940be190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f85940be190 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f859000f7a0 tx=0x7f859000fc60 comp rx=0 tx=0).stop 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 shutdown_connections 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f857806be60 0x7f857806e320 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8594149cf0 0x7f859414a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85940af700 0x7f85940afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 --2- 172.21.15.160:0/1105361138 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8580004800 0x7f85940be190 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 >> 172.21.15.160:0/1105361138 conn(0x7f8594072ec0 msgr2=0x7f859414b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:58.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 shutdown_connections 2024-05-09T07:13:58.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.099+0000 7f859bd12640 1 -- 172.21.15.160:0/1105361138 wait complete. 2024-05-09T07:13:58.111 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:13:58.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.214+0000 7fbce3c45640 1 Processor -- start 2024-05-09T07:13:58.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.214+0000 7fbce3c45640 1 -- start start 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce3c45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc1561a0 0x7fbcdc0c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce3c45640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbcdc0c3aa0 0x7fbcdc0c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce3c45640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbcdc0c29b0 con 0x7fbcdc0c7270 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce3c45640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fbcdc0c6de0 con 0x7fbcdc0c3aa0 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce3c45640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbcdc0c2c70 con 0x7fbcdc1561a0 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbcdc0c3aa0 0x7fbcdc0c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbcdc0c3aa0 0x7fbcdc0c3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58542/0 (socket says 172.21.15.160:58542) 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 -- 172.21.15.160:0/4149447998 learned_addr learned my addr 172.21.15.160:0/4149447998 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:58.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce11e0640 1 --1- 172.21.15.160:0/4149447998 >> v1:172.21.15.160:6789/0 conn(0x7fbcdc0c7270 0x7fbcdc0c76b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36406/0 (socket says 172.21.15.160:36406) 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce19e1640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc1561a0 0x7fbcdc0c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 -- 172.21.15.160:0/4149447998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc1561a0 msgr2=0x7fbcdc0c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc1561a0 0x7fbcdc0c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 -- 172.21.15.160:0/4149447998 >> v1:172.21.15.160:6789/0 conn(0x7fbcdc0c7270 legacy=0x7fbcdc0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce21e2640 1 -- 172.21.15.160:0/4149447998 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc0c6a70 con 0x7fbcdc0c3aa0 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.215+0000 7fbce19e1640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc1561a0 0x7fbcdc0c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.216+0000 7fbce21e2640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc0c3f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009870 tx=0x7fbccc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976a8bc8ca56a9f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.216+0000 7fbce09df640 1 -- 172.21.15.160:0/4149447998 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc037070 con 0x7fbcdc0c3aa0 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.216+0000 7fbce09df640 1 -- 172.21.15.160:0/4149447998 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbccc032440 con 0x7fbcdc0c3aa0 2024-05-09T07:13:58.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.216+0000 7fbce09df640 1 -- 172.21.15.160:0/4149447998 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc032810 con 0x7fbcdc0c3aa0 2024-05-09T07:13:58.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 -- 172.21.15.160:0/4149447998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc0c3aa0 msgr2=0x7fbcdc0c3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc0c3f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009870 tx=0x7fbccc02a270 comp rx=0 tx=0).stop 2024-05-09T07:13:58.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 -- 172.21.15.160:0/4149447998 shutdown_connections 2024-05-09T07:13:58.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc0c3f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 --2- 172.21.15.160:0/4149447998 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc1561a0 0x7fbcdc0c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 -- 172.21.15.160:0/4149447998 >> 172.21.15.160:0/4149447998 conn(0x7fbcdc0aa880 msgr2=0x7fbcdc0aac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 -- 172.21.15.160:0/4149447998 shutdown_connections 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.217+0000 7fbce3c45640 1 -- 172.21.15.160:0/4149447998 wait complete. 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.218+0000 7fbce3c45640 1 Processor -- start 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.218+0000 7fbce3c45640 1 -- start start 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc1ee4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc0c7270 0x7fbcdc0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 0x7fbcdc151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc156580 con 0x7fbcdc1561a0 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbcdc0c2f50 con 0x7fbcdc0c3aa0 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbcdc156700 con 0x7fbcdc0c7270 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 0x7fbcdc151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 0x7fbcdc151fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58556/0 (socket says 172.21.15.160:58556) 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 -- 172.21.15.160:0/2418594357 learned_addr learned my addr 172.21.15.160:0/2418594357 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:58.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce11e0640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc0c7270 0x7fbcdc0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce19e1640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc1ee4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 -- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc0c7270 msgr2=0x7fbcdc0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc0c7270 0x7fbcdc0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 -- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcdc0c3aa0 msgr2=0x7fbcdc1ee4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc1ee4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 -- 172.21.15.160:0/2418594357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcdc09b9a0 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce21e2640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 0x7fbcdc151fd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0097c0 tx=0x7fbccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc037040 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbccc042070 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcdc1593e0 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.219+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbccc030c90 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.220+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcdc09b480 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.221+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcdc159900 con 0x7fbcdc1561a0 2024-05-09T07:13:58.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.221+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbccc047460 con 0x7fbcdc1561a0 2024-05-09T07:13:58.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.221+0000 7fbcda7fc640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbcc806be30 0x7fbcc806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.221+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbccc0abcc0 con 0x7fbcdc1561a0 2024-05-09T07:13:58.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.221+0000 7fbce19e1640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbcc806be30 0x7fbcc806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.222+0000 7fbce19e1640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbcc806be30 0x7fbcc806e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400d620 tx=0x7fbcd4007810 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:58.227 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.226+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbccc079ce0 con 0x7fbcdc1561a0 2024-05-09T07:13:58.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.400+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fbcdc09b270 con 0x7fbcdc1561a0 2024-05-09T07:13:58.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.400+0000 7fbcda7fc640 1 -- 172.21.15.160:0/2418594357 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fbccc07dfe0 con 0x7fbcdc1561a0 2024-05-09T07:13:58.402 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:13:58.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbcc806be30 msgr2=0x7fbcc806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbcc806be30 0x7fbcc806e2f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbcd400d620 tx=0x7fbcd4007810 comp rx=0 tx=0).stop 2024-05-09T07:13:58.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 msgr2=0x7fbcdc151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 0x7fbcdc151fd0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0097c0 tx=0x7fbccc0046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 shutdown_connections 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbcc806be30 0x7fbcc806e2f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbcdc1561a0 0x7fbcdc151fd0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbcdc0c7270 0x7fbcdc0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 --2- 172.21.15.160:0/2418594357 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbcdc0c3aa0 0x7fbcdc1ee4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 >> 172.21.15.160:0/2418594357 conn(0x7fbcdc0aa880 msgr2=0x7fbcdc0c76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 shutdown_connections 2024-05-09T07:13:58.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.402+0000 7fbce3c45640 1 -- 172.21.15.160:0/2418594357 wait complete. 2024-05-09T07:13:58.415 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T07:13:58.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.514+0000 7fba6de2a640 1 Processor -- start 2024-05-09T07:13:58.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.514+0000 7fba6de2a640 1 -- start start 2024-05-09T07:13:58.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba6de2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680c4010 0x7fba681570f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba6de2a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba68157630 con 0x7fba6806fb50 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba6de2a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba681534f0 con 0x7fba680c74a0 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba6de2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fba68154100 con 0x7fba680c4010 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba677fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fba6806fb50 0x7fba680c6d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36416/0 (socket says 172.21.15.160:36416) 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba677fe640 1 -- 172.21.15.160:0/2648865832 learned_addr learned my addr 172.21.15.160:0/2648865832 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba67fff640 1 --2- 172.21.15.160:0/2648865832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680c4010 0x7fba681570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba667fc640 1 -- 172.21.15.160:0/2648865832 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2666649506 0 0) 0x7fba68157630 con 0x7fba6806fb50 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.515+0000 7fba667fc640 1 -- 172.21.15.160:0/2648865832 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fba50003520 con 0x7fba6806fb50 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.516+0000 7fba667fc640 1 -- 172.21.15.160:0/2648865832 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2182926259 0 0) 0x7fba681534f0 con 0x7fba680c74a0 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.516+0000 7fba67fff640 1 -- 172.21.15.160:0/2648865832 >> v1:172.21.15.188:6789/0 conn(0x7fba680c74a0 legacy=0x7fba681569e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.516+0000 7fba67fff640 1 -- 172.21.15.160:0/2648865832 >> v1:172.21.15.160:6789/0 conn(0x7fba6806fb50 legacy=0x7fba680c6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.516+0000 7fba67fff640 1 -- 172.21.15.160:0/2648865832 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba681537d0 con 0x7fba680c4010 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.516+0000 7fba67fff640 1 --2- 172.21.15.160:0/2648865832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680c4010 0x7fba681570f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba60009dc0 tx=0x7fba6002a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6d6b3f5382a74cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba667fc640 1 -- 172.21.15.160:0/2648865832 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60035020 con 0x7fba680c4010 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba667fc640 1 -- 172.21.15.160:0/2648865832 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba60007540 con 0x7fba680c4010 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba667fc640 1 -- 172.21.15.160:0/2648865832 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba58001670 con 0x7fba680c4010 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 -- 172.21.15.160:0/2648865832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680c4010 msgr2=0x7fba681570f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2648865832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680c4010 0x7fba681570f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fba60009dc0 tx=0x7fba6002a5b0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 -- 172.21.15.160:0/2648865832 shutdown_connections 2024-05-09T07:13:58.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2648865832 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba680c4010 0x7fba681570f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 -- 172.21.15.160:0/2648865832 >> 172.21.15.160:0/2648865832 conn(0x7fba680a9800 msgr2=0x7fba680a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:58.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 -- 172.21.15.160:0/2648865832 shutdown_connections 2024-05-09T07:13:58.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.517+0000 7fba6de2a640 1 -- 172.21.15.160:0/2648865832 wait complete. 2024-05-09T07:13:58.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.518+0000 7fba6de2a640 1 Processor -- start 2024-05-09T07:13:58.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.518+0000 7fba6de2a640 1 -- start start 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba6de2a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba6806fb50 0x7fba681e9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba6de2a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680c4010 0x7fba680a8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba6de2a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 0x7fba681ecd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba6de2a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fba68153110 con 0x7fba681ec8f0 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba6de2a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fba68153d90 con 0x7fba680c4010 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba6de2a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fba68153290 con 0x7fba6806fb50 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba67fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 0x7fba681ecd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba67fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 0x7fba681ecd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58568/0 (socket says 172.21.15.160:58568) 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.519+0000 7fba67fff640 1 -- 172.21.15.160:0/2518909828 learned_addr learned my addr 172.21.15.160:0/2518909828 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba66ffd640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680c4010 0x7fba680a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba677fe640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba6806fb50 0x7fba681e9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba67fff640 1 -- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba6806fb50 msgr2=0x7fba681e9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba67fff640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba6806fb50 0x7fba681e9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba67fff640 1 -- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680c4010 msgr2=0x7fba680a8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba67fff640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680c4010 0x7fba680a8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba67fff640 1 -- 172.21.15.160:0/2518909828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba681e0170 con 0x7fba681ec8f0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba677fe640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba6806fb50 0x7fba681e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba67fff640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 0x7fba681ecd70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba6002aba0 tx=0x7fba60004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba6003f070 con 0x7fba681ec8f0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba60004240 con 0x7fba681ec8f0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba681df8d0 con 0x7fba681ec8f0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba60035030 con 0x7fba681ec8f0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.520+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba681dfe90 con 0x7fba681ec8f0 2024-05-09T07:13:58.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.521+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba680756c0 con 0x7fba681ec8f0 2024-05-09T07:13:58.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.522+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fba60042070 con 0x7fba681ec8f0 2024-05-09T07:13:58.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.522+0000 7fba4ffff640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fba5406be30 0x7fba5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:58.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.522+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba600ac2e0 con 0x7fba681ec8f0 2024-05-09T07:13:58.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.522+0000 7fba677fe640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fba5406be30 0x7fba5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:58.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.522+0000 7fba677fe640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fba5406be30 0x7fba5406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba580017e0 tx=0x7fba58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:58.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.526+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fba6007a8d0 con 0x7fba681ec8f0 2024-05-09T07:13:58.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.715+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba68156000 con 0x7fba681ec8f0 2024-05-09T07:13:58.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.715+0000 7fba4ffff640 1 -- 172.21.15.160:0/2518909828 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fba6007a1d0 con 0x7fba681ec8f0 2024-05-09T07:13:58.717 INFO:teuthology.orchestra.run.smithi160.stderr:listed 0 entries 2024-05-09T07:13:58.719 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fba5406be30 msgr2=0x7fba5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fba5406be30 0x7fba5406e2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fba580017e0 tx=0x7fba58009040 comp rx=0 tx=0).stop 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 msgr2=0x7fba681ecd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 0x7fba681ecd70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fba6002aba0 tx=0x7fba60004a40 comp rx=0 tx=0).stop 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 shutdown_connections 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fba5406be30 0x7fba5406e2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fba680c4010 0x7fba680a8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba681ec8f0 0x7fba681ecd70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 --2- 172.21.15.160:0/2518909828 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fba6806fb50 0x7fba681e9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 >> 172.21.15.160:0/2518909828 conn(0x7fba680a9800 msgr2=0x7fba680c3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 shutdown_connections 2024-05-09T07:13:58.720 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:58.717+0000 7fba6de2a640 1 -- 172.21.15.160:0/2518909828 wait complete. 2024-05-09T07:13:58.731 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi160.front.sepia.ceph.com... 2024-05-09T07:13:58.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:58.731 DEBUG:teuthology.orchestra.run.smithi160:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-09T07:13:58.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi160.stderr:ceph-fuse[43029]: fuse finished with error 0 and tester_r 0 2024-05-09T07:13:58.763 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:13:58.854 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:13:58.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:13:58.854 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-09T07:13:58.885 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:59.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.024+0000 7f7aecb70640 1 Processor -- start 2024-05-09T07:13:59.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.024+0000 7f7aecb70640 1 -- start start 2024-05-09T07:13:59.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.025+0000 7f7aecb70640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ae8153300 0x7f7ae80c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.025+0000 7f7aecb70640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae81563b0 con 0x7f7ae80c5e20 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.025+0000 7f7aecb70640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ae8156690 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.025+0000 7f7aecb70640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f7ae80c1630 con 0x7f7ae8153300 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.025+0000 7f7ae659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ae8153300 0x7f7ae80c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae659c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ae8153300 0x7f7ae80c58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58570/0 (socket says 172.21.15.160:58570) 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae659c640 1 -- 172.21.15.160:0/2857057795 learned_addr learned my addr 172.21.15.160:0/2857057795 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae6d9d640 1 --1- 172.21.15.160:0/2857057795 >> v1:172.21.15.160:6789/0 conn(0x7f7ae80c5e20 0x7f7ae815cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36432/0 (socket says 172.21.15.160:36432) 2024-05-09T07:13:59.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae659c640 1 -- 172.21.15.160:0/2857057795 >> v1:172.21.15.188:6789/0 conn(0x7f7ae80a5fb0 legacy=0x7f7ae80bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae659c640 1 -- 172.21.15.160:0/2857057795 >> v1:172.21.15.160:6789/0 conn(0x7f7ae80c5e20 legacy=0x7f7ae815cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae659c640 1 -- 172.21.15.160:0/2857057795 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae80c1c20 con 0x7f7ae8153300 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.026+0000 7f7ae659c640 1 --2- 172.21.15.160:0/2857057795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae8153300 0x7f7ae80c58e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800bb10 tx=0x7f7ad802c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806e5c8f19463d9e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7ae559a640 1 -- 172.21.15.160:0/2857057795 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8035070 con 0x7f7ae8153300 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7ae559a640 1 -- 172.21.15.160:0/2857057795 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad8031830 con 0x7f7ae8153300 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7ae559a640 1 -- 172.21.15.160:0/2857057795 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8031c00 con 0x7f7ae8153300 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 -- 172.21.15.160:0/2857057795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae8153300 msgr2=0x7f7ae80c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 --2- 172.21.15.160:0/2857057795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae8153300 0x7f7ae80c58e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800bb10 tx=0x7f7ad802c9d0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 -- 172.21.15.160:0/2857057795 shutdown_connections 2024-05-09T07:13:59.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 --2- 172.21.15.160:0/2857057795 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae8153300 0x7f7ae80c58e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 -- 172.21.15.160:0/2857057795 >> 172.21.15.160:0/2857057795 conn(0x7f7ae80c7580 msgr2=0x7f7ae80a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:59.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 -- 172.21.15.160:0/2857057795 shutdown_connections 2024-05-09T07:13:59.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.027+0000 7f7aecb70640 1 -- 172.21.15.160:0/2857057795 wait complete. 2024-05-09T07:13:59.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 Processor -- start 2024-05-09T07:13:59.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 -- start start 2024-05-09T07:13:59.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 0x7f7ae81dc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80c5e20 0x7f7ae80c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae8153300 0x7f7ae81eb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80c2420 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae80c19d0 con 0x7f7ae8153300 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7aecb70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae80bd220 con 0x7f7ae80c5e20 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7ae659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 0x7f7ae81dc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7ae659c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 0x7f7ae81dc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58576/0 (socket says 172.21.15.160:58576) 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7ae659c640 1 -- 172.21.15.160:0/3122420239 learned_addr learned my addr 172.21.15.160:0/3122420239 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7ae5d9b640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80c5e20 0x7f7ae80c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.029+0000 7f7ae6d9d640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae8153300 0x7f7ae81eb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae659c640 1 -- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80c5e20 msgr2=0x7f7ae80c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae659c640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80c5e20 0x7f7ae80c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae659c640 1 -- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae8153300 msgr2=0x7f7ae81eb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae659c640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae8153300 0x7f7ae81eb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae659c640 1 -- 172.21.15.160:0/3122420239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae81db9a0 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae5d9b640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80c5e20 0x7f7ae80c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7ae659c640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 0x7f7ae81dc530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ad802ca10 tx=0x7f7ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8035040 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ad8030910 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8030d00 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae814fb10 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.030+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae81500d0 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.031+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae80756c0 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.031+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7ad8049060 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.032+0000 7f7adeffd640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7ac806be30 0x7f7ac806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.032+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7ad80abd00 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.032+0000 7f7ae5d9b640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7ac806be30 0x7f7ac806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.032+0000 7f7ae5d9b640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7ac806be30 0x7f7ac806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ae81e8f90 tx=0x7f7ad0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.036+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7ae81db9a0 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.211+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ae80bdab0 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.212+0000 7f7adeffd640 1 -- 172.21.15.160:0/3122420239 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f7ad8080030 con 0x7f7ae80a5fb0 2024-05-09T07:13:59.215 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:59.215 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7ac806be30 msgr2=0x7f7ac806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7ac806be30 0x7f7ac806e2f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7ae81e8f90 tx=0x7f7ad0009040 comp rx=0 tx=0).stop 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 msgr2=0x7f7ae81dc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 0x7f7ae81dc530 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7ad802ca10 tx=0x7f7ad80046c0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 shutdown_connections 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7ac806be30 0x7f7ac806e2f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ae8153300 0x7f7ae81eb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ae80c5e20 0x7f7ae80c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 --2- 172.21.15.160:0/3122420239 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ae80a5fb0 0x7f7ae81dc530 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 >> 172.21.15.160:0/3122420239 conn(0x7f7ae80c7580 msgr2=0x7f7ae806d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:59.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 shutdown_connections 2024-05-09T07:13:59.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.215+0000 7f7aecb70640 1 -- 172.21.15.160:0/3122420239 wait complete. 2024-05-09T07:13:59.228 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:13:59.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.326+0000 7f6d6bb63640 1 Processor -- start 2024-05-09T07:13:59.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.326+0000 7f6d6bb63640 1 -- start start 2024-05-09T07:13:59.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d6bb63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d64160d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d6bb63640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d64159100 con 0x7f6d64073210 2024-05-09T07:13:59.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d6bb63640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d640a7210 con 0x7f6d640c59c0 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d6bb63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6d640c2000 con 0x7f6d64155f50 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d698ff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f6d64073210 0x7f6d641589f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36440/0 (socket says 172.21.15.160:36440) 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d698ff640 1 -- 172.21.15.160:0/1215595591 learned_addr learned my addr 172.21.15.160:0/1215595591 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.327+0000 7f6d6a100640 1 --2- 172.21.15.160:0/1215595591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d64160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1727049152 0 0) 0x7f6d64159100 con 0x7f6d64073210 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d4c003520 con 0x7f6d64073210 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2386617921 0 0) 0x7f6d640a7210 con 0x7f6d640c59c0 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d64159100 con 0x7f6d640c59c0 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 121248531 0 0) 0x7f6d4c003520 con 0x7f6d64073210 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d6a100640 1 -- 172.21.15.160:0/1215595591 >> v1:172.21.15.188:6789/0 conn(0x7f6d640c59c0 legacy=0x7f6d6406c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d6a100640 1 -- 172.21.15.160:0/1215595591 >> v1:172.21.15.160:6789/0 conn(0x7f6d64073210 legacy=0x7f6d641589f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.328+0000 7f6d6a100640 1 -- 172.21.15.160:0/1215595591 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d640a74f0 con 0x7f6d64155f50 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6a100640 1 --2- 172.21.15.160:0/1215595591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d64160d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0097d0 tx=0x7f6d5c02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a9561d7404102b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c037070 con 0x7f6d64155f50 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5c02ac30 con 0x7f6d64155f50 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d688fd640 1 -- 172.21.15.160:0/1215595591 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d540019f0 con 0x7f6d64155f50 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1215595591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 msgr2=0x7f6d64160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1215595591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d64160d90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0097d0 tx=0x7f6d5c02a1a0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1215595591 shutdown_connections 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1215595591 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d64160d90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1215595591 >> 172.21.15.160:0/1215595591 conn(0x7f6d640a9c90 msgr2=0x7f6d640aa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:59.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1215595591 shutdown_connections 2024-05-09T07:13:59.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.329+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1215595591 wait complete. 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 Processor -- start 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 -- start start 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 0x7f6d6414e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d640c59c0 0x7f6d641567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d6414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64155400 con 0x7f6d64073210 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64155a30 con 0x7f6d640c59c0 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d6bb63640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6d640735f0 con 0x7f6d64155f50 2024-05-09T07:13:59.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d698ff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 0x7f6d6414e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d698ff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 0x7f6d6414e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58580/0 (socket says 172.21.15.160:58580) 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.331+0000 7f6d698ff640 1 -- 172.21.15.160:0/1948153323 learned_addr learned my addr 172.21.15.160:0/1948153323 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d690fe640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d640c59c0 0x7f6d641567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d6a100640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d6414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d698ff640 1 -- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 msgr2=0x7f6d6414bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d698ff640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d6414bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d698ff640 1 -- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d640c59c0 msgr2=0x7f6d641567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d698ff640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d640c59c0 0x7f6d641567e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d698ff640 1 -- 172.21.15.160:0/1948153323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d641ee050 con 0x7f6d64073210 2024-05-09T07:13:59.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d698ff640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 0x7f6d6414e370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400cc40 tx=0x7f6d5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54017070 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d5400ce50 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54012640 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d641ee330 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.332+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d641ee8f0 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.333+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d640756c0 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.334+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6d5401a070 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.334+0000 7f6d627fc640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6d5006be30 0x7f6d5006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.334+0000 7f6d690fe640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6d5006be30 0x7f6d5006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.334+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6d5408b120 con 0x7f6d64073210 2024-05-09T07:13:59.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.334+0000 7f6d690fe640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6d5006be30 0x7f6d5006e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d580010e0 tx=0x7f6d58009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.338+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6d54059790 con 0x7f6d64073210 2024-05-09T07:13:59.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.513+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d640c1c30 con 0x7f6d64073210 2024-05-09T07:13:59.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.514+0000 7f6d627fc640 1 -- 172.21.15.160:0/1948153323 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f6d54059090 con 0x7f6d64073210 2024-05-09T07:13:59.516 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:13:59.516 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:13:59.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6d5006be30 msgr2=0x7f6d5006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6d5006be30 0x7f6d5006e2f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6d580010e0 tx=0x7f6d58009040 comp rx=0 tx=0).stop 2024-05-09T07:13:59.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 msgr2=0x7f6d6414e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 0x7f6d6414e370 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400cc40 tx=0x7f6d5400a430 comp rx=0 tx=0).stop 2024-05-09T07:13:59.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 shutdown_connections 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6d5006be30 0x7f6d5006e2f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6d640c59c0 0x7f6d641567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6d64155f50 0x7f6d6414bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 --2- 172.21.15.160:0/1948153323 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6d64073210 0x7f6d6414e370 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 >> 172.21.15.160:0/1948153323 conn(0x7f6d640a9c90 msgr2=0x7f6d640a6dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 shutdown_connections 2024-05-09T07:13:59.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.516+0000 7f6d6bb63640 1 -- 172.21.15.160:0/1948153323 wait complete. 2024-05-09T07:13:59.531 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 Processor -- start 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 -- start start 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe61c1573a0 0x7fe61c0c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe61c06fac0 0x7fe61c0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c0714e0 con 0x7fe61c06fac0 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe61c0c78c0 con 0x7fe61c1573a0 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.633+0000 7fe622c9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c0c3af0 con 0x7fe61c0c9020 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe61c06fac0 0x7fe61c0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe61c06fac0 0x7fe61c0c3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58590/0 (socket says 172.21.15.160:58590) 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 -- 172.21.15.160:0/1207005255 learned_addr learned my addr 172.21.15.160:0/1207005255 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:59.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe620a3a640 1 --2- 172.21.15.160:0/1207005255 >> v2:172.21.15.188:3300/0 conn(0x7fe61c1573a0 0x7fe61c0c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe62123b640 1 --2- 172.21.15.160:0/1207005255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 -- 172.21.15.160:0/1207005255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 msgr2=0x7fe61c0c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 --2- 172.21.15.160:0/1207005255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 -- 172.21.15.160:0/1207005255 >> v2:172.21.15.188:3300/0 conn(0x7fe61c1573a0 msgr2=0x7fe61c0c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:13:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 --2- 172.21.15.160:0/1207005255 >> v2:172.21.15.188:3300/0 conn(0x7fe61c1573a0 0x7fe61c0c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 -- 172.21.15.160:0/1207005255 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c0c7620 con 0x7fe61c06fac0 2024-05-09T07:13:59.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.634+0000 7fe61bfff640 1 --2- 172.21.15.160:0/1207005255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c06fac0 0x7fe61c0c3f80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0ab9c0 tx=0x7fe60802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9160487c4daba2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe61b7fe640 1 -- 172.21.15.160:0/1207005255 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608037070 con 0x7fe61c06fac0 2024-05-09T07:13:59.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe61b7fe640 1 -- 172.21.15.160:0/1207005255 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe608032440 con 0x7fe61c06fac0 2024-05-09T07:13:59.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe61b7fe640 1 -- 172.21.15.160:0/1207005255 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe608032810 con 0x7fe61c06fac0 2024-05-09T07:13:59.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 -- 172.21.15.160:0/1207005255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c06fac0 msgr2=0x7fe61c0c3f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 --2- 172.21.15.160:0/1207005255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c06fac0 0x7fe61c0c3f80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0ab9c0 tx=0x7fe60802a270 comp rx=0 tx=0).stop 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 -- 172.21.15.160:0/1207005255 shutdown_connections 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 --2- 172.21.15.160:0/1207005255 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 --2- 172.21.15.160:0/1207005255 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c06fac0 0x7fe61c0c3f80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 --2- 172.21.15.160:0/1207005255 >> v2:172.21.15.188:3300/0 conn(0x7fe61c1573a0 0x7fe61c0c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 -- 172.21.15.160:0/1207005255 >> 172.21.15.160:0/1207005255 conn(0x7fe61c072a80 msgr2=0x7fe61c072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 -- 172.21.15.160:0/1207005255 shutdown_connections 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.635+0000 7fe622c9e640 1 -- 172.21.15.160:0/1207005255 wait complete. 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.636+0000 7fe622c9e640 1 Processor -- start 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.636+0000 7fe622c9e640 1 -- start start 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe622c9e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe61c06fac0 0x7fe61c1ec270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe622c9e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe622c9e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 0x7fe61c0ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe622c9e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c0abc20 con 0x7fe61c1573a0 2024-05-09T07:13:59.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe622c9e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c157780 con 0x7fe61c06fac0 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe622c9e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe61c070520 con 0x7fe61c0c9020 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe62123b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 0x7fe61c0ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe62123b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 0x7fe61c0ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58600/0 (socket says 172.21.15.160:58600) 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.637+0000 7fe62123b640 1 -- 172.21.15.160:0/712359712 learned_addr learned my addr 172.21.15.160:0/712359712 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe61bfff640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe620a3a640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe61c06fac0 0x7fe61c1ec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe62123b640 1 -- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 msgr2=0x7fe61c0ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe62123b640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe62123b640 1 -- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe61c06fac0 msgr2=0x7fe61c1ec270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe62123b640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe61c06fac0 0x7fe61c1ec270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe62123b640 1 -- 172.21.15.160:0/712359712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c0a8b60 con 0x7fe61c1573a0 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe620a3a640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe61c06fac0 0x7fe61c1ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe62123b640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 0x7fe61c0ad5f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe614007e70 tx=0x7fe61400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614014620 con 0x7fe61c1573a0 2024-05-09T07:13:59.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe614019070 con 0x7fe61c1573a0 2024-05-09T07:13:59.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c14b810 con 0x7fe61c1573a0 2024-05-09T07:13:59.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614014a20 con 0x7fe61c1573a0 2024-05-09T07:13:59.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.638+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c0a99d0 con 0x7fe61c1573a0 2024-05-09T07:13:59.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.639+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe61c075650 con 0x7fe61c1573a0 2024-05-09T07:13:59.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.640+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe614014bc0 con 0x7fe61c1573a0 2024-05-09T07:13:59.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.640+0000 7fe6197fa640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe60006be30 0x7fe60006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:13:59.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.640+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe61408ba60 con 0x7fe61c1573a0 2024-05-09T07:13:59.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.640+0000 7fe620a3a640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe60006be30 0x7fe60006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:13:59.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.640+0000 7fe620a3a640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe60006be30 0x7fe60006e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe610006fd0 tx=0x7fe610008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:13:59.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.644+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe61c0a8b60 con 0x7fe61c1573a0 2024-05-09T07:13:59.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:13:59.853+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe61c0c62a0 con 0x7fe61c1573a0 2024-05-09T07:14:00.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.519+0000 7fe6197fa640 1 -- 172.21.15.160:0/712359712 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fe61405a160 con 0x7fe61c1573a0 2024-05-09T07:14:00.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe60006be30 msgr2=0x7fe60006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:00.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe60006be30 0x7fe60006e2f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe610006fd0 tx=0x7fe610008040 comp rx=0 tx=0).stop 2024-05-09T07:14:00.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 msgr2=0x7fe61c0ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:00.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 0x7fe61c0ad5f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe614007e70 tx=0x7fe61400e4b0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 shutdown_connections 2024-05-09T07:14:00.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe60006be30 0x7fe60006e2f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe61c1573a0 0x7fe61c0ad5f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe61c0c9020 0x7fe61c0ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 --2- 172.21.15.160:0/712359712 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe61c06fac0 0x7fe61c1ec270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 >> 172.21.15.160:0/712359712 conn(0x7fe61c072a80 msgr2=0x7fe61c072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:00.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 shutdown_connections 2024-05-09T07:14:00.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.521+0000 7fe622c9e640 1 -- 172.21.15.160:0/712359712 wait complete. 2024-05-09T07:14:00.535 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-05-09T07:14:00.535 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-05-09T07:14:00.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 Processor -- start 2024-05-09T07:14:00.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 -- start start 2024-05-09T07:14:00.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3158066f90 0x7f31580c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:00.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f315814b990 0x7f315814bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:00.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31580c0e70 con 0x7f315806c3e0 2024-05-09T07:14:00.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f31580ac3b0 con 0x7f3158066f90 2024-05-09T07:14:00.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.639+0000 7f315de83640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3158155f10 con 0x7f315814b990 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3158066f90 0x7f31580c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3158066f90 0x7f31580c20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58608/0 (socket says 172.21.15.160:58608) 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 -- 172.21.15.160:0/894107207 learned_addr learned my addr 172.21.15.160:0/894107207 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3157fff640 1 --2- 172.21.15.160:0/894107207 >> v2:172.21.15.188:3300/0 conn(0x7f315814b990 0x7f315814bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 -- 172.21.15.160:0/894107207 >> v2:172.21.15.188:3300/0 conn(0x7f315814b990 msgr2=0x7f315814bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 --2- 172.21.15.160:0/894107207 >> v2:172.21.15.188:3300/0 conn(0x7f315814b990 0x7f315814bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 -- 172.21.15.160:0/894107207 >> v1:172.21.15.188:6789/0 conn(0x7f315806c3e0 legacy=0x7f31580c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.640+0000 7f3156ffd640 1 -- 172.21.15.160:0/894107207 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31580ac650 con 0x7f3158066f90 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f3156ffd640 1 --2- 172.21.15.160:0/894107207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3158066f90 0x7f31580c20f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31580675c0 tx=0x7f314c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7ca79099ca1107 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:00.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f31567fc640 1 -- 172.21.15.160:0/894107207 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c036070 con 0x7f3158066f90 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f31567fc640 1 -- 172.21.15.160:0/894107207 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f314c031600 con 0x7f3158066f90 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f31567fc640 1 -- 172.21.15.160:0/894107207 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314c0319d0 con 0x7f3158066f90 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 -- 172.21.15.160:0/894107207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3158066f90 msgr2=0x7f31580c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 --2- 172.21.15.160:0/894107207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3158066f90 0x7f31580c20f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f31580675c0 tx=0x7f314c02a6a0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 -- 172.21.15.160:0/894107207 shutdown_connections 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 --2- 172.21.15.160:0/894107207 >> v2:172.21.15.188:3300/0 conn(0x7f315814b990 0x7f315814bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 --2- 172.21.15.160:0/894107207 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3158066f90 0x7f31580c20f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 -- 172.21.15.160:0/894107207 >> 172.21.15.160:0/894107207 conn(0x7f31580c3e30 msgr2=0x7f31580c4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 -- 172.21.15.160:0/894107207 shutdown_connections 2024-05-09T07:14:00.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.641+0000 7f315de83640 1 -- 172.21.15.160:0/894107207 wait complete. 2024-05-09T07:14:00.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.642+0000 7f315de83640 1 Processor -- start 2024-05-09T07:14:00.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 -- start start 2024-05-09T07:14:00.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3158066f90 0x7f3158148af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:00.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f315806c3e0 0x7f31580726c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 0x7f31581eae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f31580c4c00 con 0x7f315814b990 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3158073780 con 0x7f3158066f90 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f315de83640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f31580c4d80 con 0x7f315806c3e0 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f3157fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 0x7f31581eae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f3157fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 0x7f31581eae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58620/0 (socket says 172.21.15.160:58620) 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.643+0000 7f3157fff640 1 -- 172.21.15.160:0/2501730846 learned_addr learned my addr 172.21.15.160:0/2501730846 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3156ffd640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f315806c3e0 0x7f31580726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f31577fe640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3158066f90 0x7f3158148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3157fff640 1 -- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f315806c3e0 msgr2=0x7f31580726c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3157fff640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f315806c3e0 0x7f31580726c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3157fff640 1 -- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3158066f90 msgr2=0x7f3158148af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3157fff640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3158066f90 0x7f3158148af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:00.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3157fff640 1 -- 172.21.15.160:0/2501730846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31581593c0 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f3157fff640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 0x7f31581eae30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f315000d5d0 tx=0x7f315000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31500125f0 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3150019070 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31500129f0 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31581550d0 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.644+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31581ee520 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.645+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3158075650 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.646+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3150012b90 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.646+0000 7f313ffff640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f314406be30 0x7f314406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.646+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f315008b570 con 0x7f315814b990 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.646+0000 7f31577fe640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f314406be30 0x7f314406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:00.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.647+0000 7f31577fe640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f314406be30 0x7f314406e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3148007fd0 tx=0x7f3148009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:00.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.650+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f31581593c0 con 0x7f315814b990 2024-05-09T07:14:00.864 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:00.863+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f31580693b0 con 0x7f315814b990 2024-05-09T07:14:01.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.523+0000 7f313ffff640 1 -- 172.21.15.160:0/2501730846 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f3150059bf0 con 0x7f315814b990 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f314406be30 msgr2=0x7f314406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f314406be30 0x7f314406e2f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3148007fd0 tx=0x7f3148009040 comp rx=0 tx=0).stop 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 msgr2=0x7f31581eae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 0x7f31581eae30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f315000d5d0 tx=0x7f315000da90 comp rx=0 tx=0).stop 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 shutdown_connections 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f314406be30 0x7f314406e2f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f315814b990 0x7f31581eae30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f315806c3e0 0x7f31580726c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 --2- 172.21.15.160:0/2501730846 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3158066f90 0x7f3158148af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.528 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 >> 172.21.15.160:0/2501730846 conn(0x7f31580c3e30 msgr2=0x7f31580abbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:01.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 shutdown_connections 2024-05-09T07:14:01.529 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.526+0000 7f315de83640 1 -- 172.21.15.160:0/2501730846 wait complete. 2024-05-09T07:14:01.540 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-05-09T07:14:01.540 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:14:01.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.641+0000 7f14428c1640 1 Processor -- start 2024-05-09T07:14:01.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.641+0000 7f14428c1640 1 -- start start 2024-05-09T07:14:01.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f14428c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 0x7f143c0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f14428c1640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f143c1563b0 con 0x7f143c153300 2024-05-09T07:14:01.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f14428c1640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f143c156690 con 0x7f143c0bc360 2024-05-09T07:14:01.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f14428c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f143c0c1630 con 0x7f143c0a5fb0 2024-05-09T07:14:01.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143bfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f143c153300 0x7f143c0c58e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36454/0 (socket says 172.21.15.160:36454) 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143bfff640 1 -- 172.21.15.160:0/1949734403 learned_addr learned my addr 172.21.15.160:0/1949734403 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3230576272 0 0) 0x7f143c1563b0 con 0x7f143c153300 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1428003520 con 0x7f143c153300 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f1433fff640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 0x7f143c0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 280527553 0 0) 0x7f1428003520 con 0x7f143c153300 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 msgr2=0x7f143c0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 0x7f143c0bbe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 >> v1:172.21.15.188:6789/0 conn(0x7f143c0bc360 legacy=0x7f143c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c0c1c20 con 0x7f143c153300 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.642+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f142c001670 con 0x7f143c153300 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 >> v1:172.21.15.160:6789/0 conn(0x7f143c153300 legacy=0x7f143c0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f1428003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 0x7f14280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f143c0c1630 con 0x7f14280039e0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f142800baa0 con 0x7f1428004800 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f142800deb0 con 0x7f143c0a5fb0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143bfff640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f1428003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f1433fff640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f1440e5e640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 0x7f14280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143bfff640 1 -- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 msgr2=0x7f14280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143bfff640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c0a5fb0 0x7f14280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143bfff640 1 -- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1428004800 msgr2=0x7f1428004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143bfff640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1428004800 0x7f1428004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.643+0000 7f143bfff640 1 -- 172.21.15.160:0/1949734403 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1428003520 con 0x7f14280039e0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f143bfff640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f1428003e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c240 tx=0x7f1428005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf67d49875d04f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c007d10 con 0x7f14280039e0 2024-05-09T07:14:01.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142c005ad0 con 0x7f14280039e0 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f143b7fe640 1 -- 172.21.15.160:0/1949734403 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c007070 con 0x7f14280039e0 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f14428c1640 1 -- 172.21.15.160:0/1949734403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 msgr2=0x7f1428003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f14428c1640 1 --2- 172.21.15.160:0/1949734403 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f1428003e00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f142c00c240 tx=0x7f1428005830 comp rx=0 tx=0).stop 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f1440e5e640 1 -- 172.21.15.160:0/1949734403 reap_dead start 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f14428c1640 1 -- 172.21.15.160:0/1949734403 shutdown_connections 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f14428c1640 1 -- 172.21.15.160:0/1949734403 >> 172.21.15.160:0/1949734403 conn(0x7f143c0c7580 msgr2=0x7f143c0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f14428c1640 1 -- 172.21.15.160:0/1949734403 shutdown_connections 2024-05-09T07:14:01.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.644+0000 7f14428c1640 1 -- 172.21.15.160:0/1949734403 wait complete. 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 Processor -- start 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 -- start start 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f143c14df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c0c5f90 0x7f143c0c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c1ebcc0 0x7f143c072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f143c0c2420 con 0x7f14280039e0 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f143c0c19d0 con 0x7f143c0c5f90 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f14428c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f143c0bc7d0 con 0x7f143c1ebcc0 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f143bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f143c14df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f143bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f143c14df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58626/0 (socket says 172.21.15.160:58626) 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.646+0000 7f143bfff640 1 -- 172.21.15.160:0/1088215432 learned_addr learned my addr 172.21.15.160:0/1088215432 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:01.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f1433fff640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c0c5f90 0x7f143c0c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f1440e5e640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c1ebcc0 0x7f143c072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f143bfff640 1 -- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c1ebcc0 msgr2=0x7f143c072f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f143bfff640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c1ebcc0 0x7f143c072f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f143bfff640 1 -- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c0c5f90 msgr2=0x7f143c0c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f143bfff640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c0c5f90 0x7f143c0c44a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f143bfff640 1 -- 172.21.15.160:0/1088215432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f143c1eb220 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f143bfff640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f143c14df90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f142c001970 tx=0x7f143c156580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c042020 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f142c004bf0 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f143c0c6470 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142c007d10 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.647+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f143c14e4d0 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.648+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f143c1eb010 con 0x7f14280039e0 2024-05-09T07:14:01.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.649+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f142c045020 con 0x7f14280039e0 2024-05-09T07:14:01.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.649+0000 7f14397fa640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f141c06be30 0x7f141c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.649+0000 7f1433fff640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f141c06be30 0x7f141c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.649+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f142c0ab7c0 con 0x7f14280039e0 2024-05-09T07:14:01.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.650+0000 7f1433fff640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f141c06be30 0x7f141c06e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f143c0717f0 tx=0x7f1424008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:01.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.654+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f142c04c070 con 0x7f14280039e0 2024-05-09T07:14:01.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.866+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f143c0756c0 con 0x7f14280039e0 2024-05-09T07:14:01.868 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.866+0000 7f14397fa640 1 -- 172.21.15.160:0/1088215432 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f142c079e80 con 0x7f14280039e0 2024-05-09T07:14:01.869 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:01.869 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:14:00.864206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:14:01.870 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-05-09T07:14:01.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f141c06be30 msgr2=0x7f141c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.870 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f141c06be30 0x7f141c06e2f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f143c0717f0 tx=0x7f1424008040 comp rx=0 tx=0).stop 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 msgr2=0x7f143c14df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f143c14df90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f142c001970 tx=0x7f143c156580 comp rx=0 tx=0).stop 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 shutdown_connections 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f141c06be30 0x7f141c06e2f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f143c0c5f90 0x7f143c0c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f143c1ebcc0 0x7f143c072f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 --2- 172.21.15.160:0/1088215432 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f14280039e0 0x7f143c14df90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 >> 172.21.15.160:0/1088215432 conn(0x7f143c0c7580 msgr2=0x7f143c150b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 shutdown_connections 2024-05-09T07:14:01.871 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.868+0000 7f14428c1640 1 -- 172.21.15.160:0/1088215432 wait complete. 2024-05-09T07:14:01.881 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:14:01.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.980+0000 7f8fa4516640 1 Processor -- start 2024-05-09T07:14:01.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.980+0000 7f8fa4516640 1 -- start start 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa4516640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c06e340 0x7f8f9c072dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa4516640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9c0c25e0 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa4516640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f9c0c28c0 con 0x7f8f9c152a10 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa4516640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c073300 con 0x7f8f9c06e340 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa1ab1640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8f9c0c33c0 0x7f8f9c0c3df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36470/0 (socket says 172.21.15.160:36470) 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa1ab1640 1 -- 172.21.15.160:0/2574384503 learned_addr learned my addr 172.21.15.160:0/2574384503 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:01.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa2ab3640 1 --2- 172.21.15.160:0/2574384503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c06e340 0x7f8f9c072dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa12b0640 1 -- 172.21.15.160:0/2574384503 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3778065262 0 0) 0x7f8f9c0c25e0 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa12b0640 1 -- 172.21.15.160:0/2574384503 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f88003520 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa2ab3640 1 -- 172.21.15.160:0/2574384503 >> v1:172.21.15.188:6789/0 conn(0x7f8f9c152a10 legacy=0x7f8f9c071820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa12b0640 1 -- 172.21.15.160:0/2574384503 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1729525455 0 0) 0x7f8f9c0c28c0 con 0x7f8f9c152a10 2024-05-09T07:14:01.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa2ab3640 1 -- 172.21.15.160:0/2574384503 >> v1:172.21.15.160:6789/0 conn(0x7f8f9c0c33c0 legacy=0x7f8f9c0c3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.981+0000 7f8fa2ab3640 1 -- 172.21.15.160:0/2574384503 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c0c2ba0 con 0x7f8f9c06e340 2024-05-09T07:14:01.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.982+0000 7f8fa2ab3640 1 --2- 172.21.15.160:0/2574384503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c06e340 0x7f8f9c072dc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009dc0 tx=0x7f8f9402a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d82e8f04d0a854af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:01.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.982+0000 7f8fa12b0640 1 -- 172.21.15.160:0/2574384503 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94007740 con 0x7f8f9c06e340 2024-05-09T07:14:01.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.982+0000 7f8fa12b0640 1 -- 172.21.15.160:0/2574384503 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f940078e0 con 0x7f8f9c06e340 2024-05-09T07:14:01.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.982+0000 7f8fa12b0640 1 -- 172.21.15.160:0/2574384503 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94007c90 con 0x7f8f9c06e340 2024-05-09T07:14:01.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 -- 172.21.15.160:0/2574384503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c06e340 msgr2=0x7f8f9c072dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 --2- 172.21.15.160:0/2574384503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c06e340 0x7f8f9c072dc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009dc0 tx=0x7f8f9402a6d0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 -- 172.21.15.160:0/2574384503 shutdown_connections 2024-05-09T07:14:01.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 --2- 172.21.15.160:0/2574384503 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c06e340 0x7f8f9c072dc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 -- 172.21.15.160:0/2574384503 >> 172.21.15.160:0/2574384503 conn(0x7f8f9c06b550 msgr2=0x7f8f9c06b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:01.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 -- 172.21.15.160:0/2574384503 shutdown_connections 2024-05-09T07:14:01.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.983+0000 7f8fa4516640 1 -- 172.21.15.160:0/2574384503 wait complete. 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.984+0000 7f8fa4516640 1 Processor -- start 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.984+0000 7f8fa4516640 1 -- start start 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa4516640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f9c06e340 0x7f8f9c1e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa4516640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 0x7f8f9c1ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa4516640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c152a10 0x7f8f9c1ed060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa4516640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c071f30 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa4516640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8f9c152df0 con 0x7f8f9c06e340 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa4516640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8f9c0c4500 con 0x7f8f9c152a10 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 0x7f8f9c1ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 0x7f8f9c1ecb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58628/0 (socket says 172.21.15.160:58628) 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 -- 172.21.15.160:0/196460219 learned_addr learned my addr 172.21.15.160:0/196460219 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:01.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa2ab3640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c152a10 0x7f8f9c1ed060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa22b2640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f9c06e340 0x7f8f9c1e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 -- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c152a10 msgr2=0x7f8f9c1ed060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c152a10 0x7f8f9c1ed060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 -- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f9c06e340 msgr2=0x7f8f9c1e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f9c06e340 0x7f8f9c1e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 -- 172.21.15.160:0/196460219 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9c14f590 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa22b2640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f9c06e340 0x7f8f9c1e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8fa1ab1640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 0x7f8f9c1ecb20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00c500 tx=0x7f8f8c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.985+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c017070 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.986+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f8c005ce0 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.986+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f9c14f870 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.986+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8c0072f0 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.986+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9c1ec320 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.987+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f9c14fd90 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.987+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8f8c01a070 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.987+0000 7f8f92ffd640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8f8006be60 0x7f8f8006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:01.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.988+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8f8c08a4c0 con 0x7f8f9c0c33c0 2024-05-09T07:14:01.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.988+0000 7f8fa22b2640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8f8006be60 0x7f8f8006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:01.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.988+0000 7f8fa22b2640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8f8006be60 0x7f8f8006e320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f98000f60 tx=0x7f8f98009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:01.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:01.992+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8f9c14fd90 con 0x7f8f9c0c33c0 2024-05-09T07:14:02.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.168+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f9c144400 con 0x7f8f9c0c33c0 2024-05-09T07:14:02.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.170+0000 7f8f92ffd640 1 -- 172.21.15.160:0/196460219 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f8f9c144400 con 0x7f8f9c0c33c0 2024-05-09T07:14:02.172 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:02.172 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8f8006be60 msgr2=0x7f8f8006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8f8006be60 0x7f8f8006e320 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8f98000f60 tx=0x7f8f98009040 comp rx=0 tx=0).stop 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 msgr2=0x7f8f9c1ecb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 0x7f8f9c1ecb20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8f8c00c500 tx=0x7f8f8c00c9c0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 shutdown_connections 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8f8006be60 0x7f8f8006e320 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8f9c152a10 0x7f8f9c1ed060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8f9c0c33c0 0x7f8f9c1ecb20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 --2- 172.21.15.160:0/196460219 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8f9c06e340 0x7f8f9c1e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 >> 172.21.15.160:0/196460219 conn(0x7f8f9c06b550 msgr2=0x7f8f9c147330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:02.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 shutdown_connections 2024-05-09T07:14:02.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.172+0000 7f8fa4516640 1 -- 172.21.15.160:0/196460219 wait complete. 2024-05-09T07:14:02.185 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-05-09T07:14:02.185 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:14:02.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa44df22640 1 Processor -- start 2024-05-09T07:14:02.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa44df22640 1 -- start start 2024-05-09T07:14:02.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa44df22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806e400 0x7fa4480c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa44df22640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa448155bf0 con 0x7fa44806b1f0 2024-05-09T07:14:02.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa44df22640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa448155ed0 con 0x7fa44806e7e0 2024-05-09T07:14:02.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa44df22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa44806ec70 con 0x7fa44806e400 2024-05-09T07:14:02.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.285+0000 7fa43effd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa44806b1f0 0x7fa448155350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36482/0 (socket says 172.21.15.160:36482) 2024-05-09T07:14:02.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa43effd640 1 -- 172.21.15.160:0/3025681137 learned_addr learned my addr 172.21.15.160:0/3025681137 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa4477fe640 1 --2- 172.21.15.160:0/3025681137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806e400 0x7fa4480c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa446ffd640 1 -- 172.21.15.160:0/3025681137 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3418226270 0 0) 0x7fa448155bf0 con 0x7fa44806b1f0 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa446ffd640 1 -- 172.21.15.160:0/3025681137 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa434003520 con 0x7fa44806b1f0 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa446ffd640 1 -- 172.21.15.160:0/3025681137 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2042227071 0 0) 0x7fa448155ed0 con 0x7fa44806e7e0 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa4477fe640 1 -- 172.21.15.160:0/3025681137 >> v1:172.21.15.188:6789/0 conn(0x7fa44806e7e0 legacy=0x7fa448159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa4477fe640 1 -- 172.21.15.160:0/3025681137 >> v1:172.21.15.160:6789/0 conn(0x7fa44806b1f0 legacy=0x7fa448155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.286+0000 7fa4477fe640 1 -- 172.21.15.160:0/3025681137 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481561b0 con 0x7fa44806e400 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa4477fe640 1 --2- 172.21.15.160:0/3025681137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806e400 0x7fa4480c1cc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4380097d0 tx=0x7fa43802a0e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34555241c9c2460f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa446ffd640 1 -- 172.21.15.160:0/3025681137 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa438037070 con 0x7fa44806e400 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa446ffd640 1 -- 172.21.15.160:0/3025681137 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43802ab70 con 0x7fa44806e400 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa446ffd640 1 -- 172.21.15.160:0/3025681137 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa430001670 con 0x7fa44806e400 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa44df22640 1 -- 172.21.15.160:0/3025681137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806e400 msgr2=0x7fa4480c1cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa44df22640 1 --2- 172.21.15.160:0/3025681137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806e400 0x7fa4480c1cc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa4380097d0 tx=0x7fa43802a0e0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa44df22640 1 -- 172.21.15.160:0/3025681137 shutdown_connections 2024-05-09T07:14:02.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa44df22640 1 --2- 172.21.15.160:0/3025681137 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806e400 0x7fa4480c1cc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.287+0000 7fa44df22640 1 -- 172.21.15.160:0/3025681137 >> 172.21.15.160:0/3025681137 conn(0x7fa4480c76e0 msgr2=0x7fa4480c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:02.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.288+0000 7fa44df22640 1 -- 172.21.15.160:0/3025681137 shutdown_connections 2024-05-09T07:14:02.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.288+0000 7fa44df22640 1 -- 172.21.15.160:0/3025681137 wait complete. 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 Processor -- start 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 -- start start 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806b1f0 0x7fa448155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa44806e400 0x7fa448153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 0x7fa4481544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa448159850 con 0x7fa44806e7e0 2024-05-09T07:14:02.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa448158c20 con 0x7fa44806e400 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.289+0000 7fa44df22640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa4480c2e60 con 0x7fa44806b1f0 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 0x7fa4481544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 0x7fa4481544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:58632/0 (socket says 172.21.15.160:58632) 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 -- 172.21.15.160:0/103593835 learned_addr learned my addr 172.21.15.160:0/103593835 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa43effd640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa44806e400 0x7fa448153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa4477fe640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806b1f0 0x7fa448155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 -- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806b1f0 msgr2=0x7fa448155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806b1f0 0x7fa448155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 -- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa44806e400 msgr2=0x7fa448153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa44806e400 0x7fa448153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 -- 172.21.15.160:0/103593835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4481531c0 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa447fff640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 0x7fa4481544a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c780 tx=0x7fa44000cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440003b70 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa440003d10 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4481eadc0 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa440005270 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.290+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa448153480 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.291+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4480756c0 con 0x7fa44806e7e0 2024-05-09T07:14:02.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.292+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa440005410 con 0x7fa44806e7e0 2024-05-09T07:14:02.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.292+0000 7fa444ff9640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa42806be60 0x7fa42806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.292+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa44008ae90 con 0x7fa44806e7e0 2024-05-09T07:14:02.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.292+0000 7fa4477fe640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa42806be60 0x7fa42806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.293+0000 7fa4477fe640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa42806be60 0x7fa42806e320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa438034000 tx=0x7fa43802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.296+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa440058ec0 con 0x7fa44806e7e0 2024-05-09T07:14:02.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.468+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa448152f70 con 0x7fa42806be60 2024-05-09T07:14:02.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.476+0000 7fa444ff9640 1 -- 172.21.15.160:0/103593835 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7fa448152f70 con 0x7fa42806be60 2024-05-09T07:14:02.484 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:02.486 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:14:02.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.479+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa42806be60 msgr2=0x7fa42806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.479+0000 7fa44df22640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa42806be60 0x7fa42806e320 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fa438034000 tx=0x7fa43802e000 comp rx=0 tx=0).stop 2024-05-09T07:14:02.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.479+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 msgr2=0x7fa4481544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.479+0000 7fa44df22640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 0x7fa4481544a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa44000c780 tx=0x7fa44000cb80 comp rx=0 tx=0).stop 2024-05-09T07:14:02.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.479+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 shutdown_connections 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa42806be60 0x7fa42806e320 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa44806e400 0x7fa448153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa44806e7e0 0x7fa4481544a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 --2- 172.21.15.160:0/103593835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa44806b1f0 0x7fa448155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 >> 172.21.15.160:0/103593835 conn(0x7fa4480c76e0 msgr2=0x7fa44814e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 shutdown_connections 2024-05-09T07:14:02.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.480+0000 7fa44df22640 1 -- 172.21.15.160:0/103593835 wait complete. 2024-05-09T07:14:02.493 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-05-09T07:14:01.182022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3383296,"data_stored":2048878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001990"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199324+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:13:52.199324+0000","last_peered":"2024-05-09T07:13:52.199324+0000","last_clean":"2024-05-09T07:13:52.199324+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:13:52.199324+0000","last_undegraded":"2024-05-09T07:13:52.199324+0000","last_fullsized":"2024-05-09T07:13:52.199324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491364+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:13:51.491364+0000","last_peered":"2024-05-09T07:13:51.491364+0000","last_clean":"2024-05-09T07:13:51.491364+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:13:51.491364+0000","last_undegraded":"2024-05-09T07:13:51.491364+0000","last_fullsized":"2024-05-09T07:13:51.491364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490779+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:13:51.490779+0000","last_peered":"2024-05-09T07:13:51.490779+0000","last_clean":"2024-05-09T07:13:51.490779+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:13:51.490779+0000","last_undegraded":"2024-05-09T07:13:51.490779+0000","last_fullsized":"2024-05-09T07:13:51.490779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491562+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:13:51.491562+0000","last_peered":"2024-05-09T07:13:51.491562+0000","last_clean":"2024-05-09T07:13:51.491562+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:13:51.491562+0000","last_undegraded":"2024-05-09T07:13:51.491562+0000","last_fullsized":"2024-05-09T07:13:51.491562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199187+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:13:52.199187+0000","last_peered":"2024-05-09T07:13:52.199187+0000","last_clean":"2024-05-09T07:13:52.199187+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:13:52.199187+0000","last_undegraded":"2024-05-09T07:13:52.199187+0000","last_fullsized":"2024-05-09T07:13:52.199187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800899+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:13:51.800899+0000","last_peered":"2024-05-09T07:13:51.800899+0000","last_clean":"2024-05-09T07:13:51.800899+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:13:51.800899+0000","last_undegraded":"2024-05-09T07:13:51.800899+0000","last_fullsized":"2024-05-09T07:13:51.800899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698480+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:13:51.698480+0000","last_peered":"2024-05-09T07:13:51.698480+0000","last_clean":"2024-05-09T07:13:51.698480+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:13:51.698480+0000","last_undegraded":"2024-05-09T07:13:51.698480+0000","last_fullsized":"2024-05-09T07:13:51.698480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202587+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:13:52.202587+0000","last_peered":"2024-05-09T07:13:52.202587+0000","last_clean":"2024-05-09T07:13:52.202587+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:13:52.202587+0000","last_undegraded":"2024-05-09T07:13:52.202587+0000","last_fullsized":"2024-05-09T07:13:52.202587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801107+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:13:51.801107+0000","last_peered":"2024-05-09T07:13:51.801107+0000","last_clean":"2024-05-09T07:13:51.801107+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:13:51.801107+0000","last_undegraded":"2024-05-09T07:13:51.801107+0000","last_fullsized":"2024-05-09T07:13:51.801107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698221+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:13:51.698221+0000","last_peered":"2024-05-09T07:13:51.698221+0000","last_clean":"2024-05-09T07:13:51.698221+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:13:51.698221+0000","last_undegraded":"2024-05-09T07:13:51.698221+0000","last_fullsized":"2024-05-09T07:13:51.698221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202703+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:13:52.202703+0000","last_peered":"2024-05-09T07:13:52.202703+0000","last_clean":"2024-05-09T07:13:52.202703+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:13:52.202703+0000","last_undegraded":"2024-05-09T07:13:52.202703+0000","last_fullsized":"2024-05-09T07:13:52.202703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800950+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:13:51.800950+0000","last_peered":"2024-05-09T07:13:51.800950+0000","last_clean":"2024-05-09T07:13:51.800950+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:13:51.800950+0000","last_undegraded":"2024-05-09T07:13:51.800950+0000","last_fullsized":"2024-05-09T07:13:51.800950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201596+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:13:52.201596+0000","last_peered":"2024-05-09T07:13:52.201596+0000","last_clean":"2024-05-09T07:13:52.201596+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:13:52.201596+0000","last_undegraded":"2024-05-09T07:13:52.201596+0000","last_fullsized":"2024-05-09T07:13:52.201596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698232+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:13:51.698232+0000","last_peered":"2024-05-09T07:13:51.698232+0000","last_clean":"2024-05-09T07:13:51.698232+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:13:51.698232+0000","last_undegraded":"2024-05-09T07:13:51.698232+0000","last_fullsized":"2024-05-09T07:13:51.698232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801528+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:13:51.801528+0000","last_peered":"2024-05-09T07:13:51.801528+0000","last_clean":"2024-05-09T07:13:51.801528+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:13:51.801528+0000","last_undegraded":"2024-05-09T07:13:51.801528+0000","last_fullsized":"2024-05-09T07:13:51.801528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491149+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:13:51.491149+0000","last_peered":"2024-05-09T07:13:51.491149+0000","last_clean":"2024-05-09T07:13:51.491149+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:13:51.491149+0000","last_undegraded":"2024-05-09T07:13:51.491149+0000","last_fullsized":"2024-05-09T07:13:51.491149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491381+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:13:51.491381+0000","last_peered":"2024-05-09T07:13:51.491381+0000","last_clean":"2024-05-09T07:13:51.491381+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:13:51.491381+0000","last_undegraded":"2024-05-09T07:13:51.491381+0000","last_fullsized":"2024-05-09T07:13:51.491381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801475+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:13:51.801475+0000","last_peered":"2024-05-09T07:13:51.801475+0000","last_clean":"2024-05-09T07:13:51.801475+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:13:51.801475+0000","last_undegraded":"2024-05-09T07:13:51.801475+0000","last_fullsized":"2024-05-09T07:13:51.801475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491169+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:13:51.491169+0000","last_peered":"2024-05-09T07:13:51.491169+0000","last_clean":"2024-05-09T07:13:51.491169+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:13:51.491169+0000","last_undegraded":"2024-05-09T07:13:51.491169+0000","last_fullsized":"2024-05-09T07:13:51.491169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490742+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:13:51.490742+0000","last_peered":"2024-05-09T07:13:51.490742+0000","last_clean":"2024-05-09T07:13:51.490742+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:13:51.490742+0000","last_undegraded":"2024-05-09T07:13:51.490742+0000","last_fullsized":"2024-05-09T07:13:51.490742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491961+0000","last_change":"2024-05-09T07:13:49.938952+0000","last_active":"2024-05-09T07:13:51.491961+0000","last_peered":"2024-05-09T07:13:51.491961+0000","last_clean":"2024-05-09T07:13:51.491961+0000","last_became_active":"2024-05-09T07:13:49.938552+0000","last_became_peered":"2024-05-09T07:13:49.938552+0000","last_unstale":"2024-05-09T07:13:51.491961+0000","last_undegraded":"2024-05-09T07:13:51.491961+0000","last_fullsized":"2024-05-09T07:13:51.491961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801398+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:13:51.801398+0000","last_peered":"2024-05-09T07:13:51.801398+0000","last_clean":"2024-05-09T07:13:51.801398+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:13:51.801398+0000","last_undegraded":"2024-05-09T07:13:51.801398+0000","last_fullsized":"2024-05-09T07:13:51.801398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697811+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:13:51.697811+0000","last_peered":"2024-05-09T07:13:51.697811+0000","last_clean":"2024-05-09T07:13:51.697811+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:13:51.697811+0000","last_undegraded":"2024-05-09T07:13:51.697811+0000","last_fullsized":"2024-05-09T07:13:51.697811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801577+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:13:51.801577+0000","last_peered":"2024-05-09T07:13:51.801577+0000","last_clean":"2024-05-09T07:13:51.801577+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:13:51.801577+0000","last_undegraded":"2024-05-09T07:13:51.801577+0000","last_fullsized":"2024-05-09T07:13:51.801577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199329+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:13:52.199329+0000","last_peered":"2024-05-09T07:13:52.199329+0000","last_clean":"2024-05-09T07:13:52.199329+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:13:52.199329+0000","last_undegraded":"2024-05-09T07:13:52.199329+0000","last_fullsized":"2024-05-09T07:13:52.199329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202341+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:13:52.202341+0000","last_peered":"2024-05-09T07:13:52.202341+0000","last_clean":"2024-05-09T07:13:52.202341+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:13:52.202341+0000","last_undegraded":"2024-05-09T07:13:52.202341+0000","last_fullsized":"2024-05-09T07:13:52.202341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800919+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:13:51.800919+0000","last_peered":"2024-05-09T07:13:51.800919+0000","last_clean":"2024-05-09T07:13:51.800919+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:13:51.800919+0000","last_undegraded":"2024-05-09T07:13:51.800919+0000","last_fullsized":"2024-05-09T07:13:51.800919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492207+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:13:51.492207+0000","last_peered":"2024-05-09T07:13:51.492207+0000","last_clean":"2024-05-09T07:13:51.492207+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:13:51.492207+0000","last_undegraded":"2024-05-09T07:13:51.492207+0000","last_fullsized":"2024-05-09T07:13:51.492207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491259+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:13:51.491259+0000","last_peered":"2024-05-09T07:13:51.491259+0000","last_clean":"2024-05-09T07:13:51.491259+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:13:51.491259+0000","last_undegraded":"2024-05-09T07:13:51.491259+0000","last_fullsized":"2024-05-09T07:13:51.491259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490763+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:13:51.490763+0000","last_peered":"2024-05-09T07:13:51.490763+0000","last_clean":"2024-05-09T07:13:51.490763+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:13:51.490763+0000","last_undegraded":"2024-05-09T07:13:51.490763+0000","last_fullsized":"2024-05-09T07:13:51.490763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491690+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:13:51.491690+0000","last_peered":"2024-05-09T07:13:51.491690+0000","last_clean":"2024-05-09T07:13:51.491690+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:13:51.491690+0000","last_undegraded":"2024-05-09T07:13:51.491690+0000","last_fullsized":"2024-05-09T07:13:51.491690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698566+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:13:51.698566+0000","last_peered":"2024-05-09T07:13:51.698566+0000","last_clean":"2024-05-09T07:13:51.698566+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:13:51.698566+0000","last_undegraded":"2024-05-09T07:13:51.698566+0000","last_fullsized":"2024-05-09T07:13:51.698566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198963+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:13:52.198963+0000","last_peered":"2024-05-09T07:13:52.198963+0000","last_clean":"2024-05-09T07:13:52.198963+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:13:52.198963+0000","last_undegraded":"2024-05-09T07:13:52.198963+0000","last_fullsized":"2024-05-09T07:13:52.198963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202806+0000","last_change":"2024-05-09T07:13:45.484793+0000","last_active":"2024-05-09T07:13:52.202806+0000","last_peered":"2024-05-09T07:13:52.202806+0000","last_clean":"2024-05-09T07:13:52.202806+0000","last_became_active":"2024-05-09T07:13:45.484248+0000","last_became_peered":"2024-05-09T07:13:45.484248+0000","last_unstale":"2024-05-09T07:13:52.202806+0000","last_undegraded":"2024-05-09T07:13:52.202806+0000","last_fullsized":"2024-05-09T07:13:52.202806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491769+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:13:51.491769+0000","last_peered":"2024-05-09T07:13:51.491769+0000","last_clean":"2024-05-09T07:13:51.491769+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:13:51.491769+0000","last_undegraded":"2024-05-09T07:13:51.491769+0000","last_fullsized":"2024-05-09T07:13:51.491769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491114+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:13:51.491114+0000","last_peered":"2024-05-09T07:13:51.491114+0000","last_clean":"2024-05-09T07:13:51.491114+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:13:51.491114+0000","last_undegraded":"2024-05-09T07:13:51.491114+0000","last_fullsized":"2024-05-09T07:13:51.491114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801532+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:13:51.801532+0000","last_peered":"2024-05-09T07:13:51.801532+0000","last_clean":"2024-05-09T07:13:51.801532+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:13:51.801532+0000","last_undegraded":"2024-05-09T07:13:51.801532+0000","last_fullsized":"2024-05-09T07:13:51.801532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202356+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:13:52.202356+0000","last_peered":"2024-05-09T07:13:52.202356+0000","last_clean":"2024-05-09T07:13:52.202356+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:13:52.202356+0000","last_undegraded":"2024-05-09T07:13:52.202356+0000","last_fullsized":"2024-05-09T07:13:52.202356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491646+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:13:51.491646+0000","last_peered":"2024-05-09T07:13:51.491646+0000","last_clean":"2024-05-09T07:13:51.491646+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:13:51.491646+0000","last_undegraded":"2024-05-09T07:13:51.491646+0000","last_fullsized":"2024-05-09T07:13:51.491646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490640+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:13:51.490640+0000","last_peered":"2024-05-09T07:13:51.490640+0000","last_clean":"2024-05-09T07:13:51.490640+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:13:51.490640+0000","last_undegraded":"2024-05-09T07:13:51.490640+0000","last_fullsized":"2024-05-09T07:13:51.490640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490075+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:13:51.490075+0000","last_peered":"2024-05-09T07:13:51.490075+0000","last_clean":"2024-05-09T07:13:51.490075+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:13:51.490075+0000","last_undegraded":"2024-05-09T07:13:51.490075+0000","last_fullsized":"2024-05-09T07:13:51.490075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492028+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:13:51.492028+0000","last_peered":"2024-05-09T07:13:51.492028+0000","last_clean":"2024-05-09T07:13:51.492028+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:13:51.492028+0000","last_undegraded":"2024-05-09T07:13:51.492028+0000","last_fullsized":"2024-05-09T07:13:51.492028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198795+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:13:52.198795+0000","last_peered":"2024-05-09T07:13:52.198795+0000","last_clean":"2024-05-09T07:13:52.198795+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:13:52.198795+0000","last_undegraded":"2024-05-09T07:13:52.198795+0000","last_fullsized":"2024-05-09T07:13:52.198795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697830+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:13:51.697830+0000","last_peered":"2024-05-09T07:13:51.697830+0000","last_clean":"2024-05-09T07:13:51.697830+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:13:51.697830+0000","last_undegraded":"2024-05-09T07:13:51.697830+0000","last_fullsized":"2024-05-09T07:13:51.697830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199305+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:13:52.199305+0000","last_peered":"2024-05-09T07:13:52.199305+0000","last_clean":"2024-05-09T07:13:52.199305+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:13:52.199305+0000","last_undegraded":"2024-05-09T07:13:52.199305+0000","last_fullsized":"2024-05-09T07:13:52.199305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491484+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:13:51.491484+0000","last_peered":"2024-05-09T07:13:51.491484+0000","last_clean":"2024-05-09T07:13:51.491484+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:13:51.491484+0000","last_undegraded":"2024-05-09T07:13:51.491484+0000","last_fullsized":"2024-05-09T07:13:51.491484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202232+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:13:52.202232+0000","last_peered":"2024-05-09T07:13:52.202232+0000","last_clean":"2024-05-09T07:13:52.202232+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:13:52.202232+0000","last_undegraded":"2024-05-09T07:13:52.202232+0000","last_fullsized":"2024-05-09T07:13:52.202232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697989+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:13:51.697989+0000","last_peered":"2024-05-09T07:13:51.697989+0000","last_clean":"2024-05-09T07:13:51.697989+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:13:51.697989+0000","last_undegraded":"2024-05-09T07:13:51.697989+0000","last_fullsized":"2024-05-09T07:13:51.697989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698344+0000","last_change":"2024-05-09T07:13:45.490251+0000","last_active":"2024-05-09T07:13:51.698344+0000","last_peered":"2024-05-09T07:13:51.698344+0000","last_clean":"2024-05-09T07:13:51.698344+0000","last_became_active":"2024-05-09T07:13:45.489707+0000","last_became_peered":"2024-05-09T07:13:45.489707+0000","last_unstale":"2024-05-09T07:13:51.698344+0000","last_undegraded":"2024-05-09T07:13:51.698344+0000","last_fullsized":"2024-05-09T07:13:51.698344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490413+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:13:51.490413+0000","last_peered":"2024-05-09T07:13:51.490413+0000","last_clean":"2024-05-09T07:13:51.490413+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:13:51.490413+0000","last_undegraded":"2024-05-09T07:13:51.490413+0000","last_fullsized":"2024-05-09T07:13:51.490413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201756+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:13:52.201756+0000","last_peered":"2024-05-09T07:13:52.201756+0000","last_clean":"2024-05-09T07:13:52.201756+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:13:52.201756+0000","last_undegraded":"2024-05-09T07:13:52.201756+0000","last_fullsized":"2024-05-09T07:13:52.201756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202663+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:13:52.202663+0000","last_peered":"2024-05-09T07:13:52.202663+0000","last_clean":"2024-05-09T07:13:52.202663+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:13:52.202663+0000","last_undegraded":"2024-05-09T07:13:52.202663+0000","last_fullsized":"2024-05-09T07:13:52.202663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490395+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:13:51.490395+0000","last_peered":"2024-05-09T07:13:51.490395+0000","last_clean":"2024-05-09T07:13:51.490395+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:13:51.490395+0000","last_undegraded":"2024-05-09T07:13:51.490395+0000","last_fullsized":"2024-05-09T07:13:51.490395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199283+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:13:52.199283+0000","last_peered":"2024-05-09T07:13:52.199283+0000","last_clean":"2024-05-09T07:13:52.199283+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:13:52.199283+0000","last_undegraded":"2024-05-09T07:13:52.199283+0000","last_fullsized":"2024-05-09T07:13:52.199283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698652+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:13:51.698652+0000","last_peered":"2024-05-09T07:13:51.698652+0000","last_clean":"2024-05-09T07:13:51.698652+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:13:51.698652+0000","last_undegraded":"2024-05-09T07:13:51.698652+0000","last_fullsized":"2024-05-09T07:13:51.698652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490223+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:13:51.490223+0000","last_peered":"2024-05-09T07:13:51.490223+0000","last_clean":"2024-05-09T07:13:51.490223+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:13:51.490223+0000","last_undegraded":"2024-05-09T07:13:51.490223+0000","last_fullsized":"2024-05-09T07:13:51.490223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201384+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:13:52.201384+0000","last_peered":"2024-05-09T07:13:52.201384+0000","last_clean":"2024-05-09T07:13:52.201384+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:13:52.201384+0000","last_undegraded":"2024-05-09T07:13:52.201384+0000","last_fullsized":"2024-05-09T07:13:52.201384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698489+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:13:51.698489+0000","last_peered":"2024-05-09T07:13:51.698489+0000","last_clean":"2024-05-09T07:13:51.698489+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:13:51.698489+0000","last_undegraded":"2024-05-09T07:13:51.698489+0000","last_fullsized":"2024-05-09T07:13:51.698489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801555+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:13:51.801555+0000","last_peered":"2024-05-09T07:13:51.801555+0000","last_clean":"2024-05-09T07:13:51.801555+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:13:51.801555+0000","last_undegraded":"2024-05-09T07:13:51.801555+0000","last_fullsized":"2024-05-09T07:13:51.801555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800811+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:13:51.800811+0000","last_peered":"2024-05-09T07:13:51.800811+0000","last_clean":"2024-05-09T07:13:51.800811+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:13:51.800811+0000","last_undegraded":"2024-05-09T07:13:51.800811+0000","last_fullsized":"2024-05-09T07:13:51.800811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698054+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:13:51.698054+0000","last_peered":"2024-05-09T07:13:51.698054+0000","last_clean":"2024-05-09T07:13:51.698054+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:13:51.698054+0000","last_undegraded":"2024-05-09T07:13:51.698054+0000","last_fullsized":"2024-05-09T07:13:51.698054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489838+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:13:51.489838+0000","last_peered":"2024-05-09T07:13:51.489838+0000","last_clean":"2024-05-09T07:13:51.489838+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:13:51.489838+0000","last_undegraded":"2024-05-09T07:13:51.489838+0000","last_fullsized":"2024-05-09T07:13:51.489838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199184+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:13:52.199184+0000","last_peered":"2024-05-09T07:13:52.199184+0000","last_clean":"2024-05-09T07:13:52.199184+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:13:52.199184+0000","last_undegraded":"2024-05-09T07:13:52.199184+0000","last_fullsized":"2024-05-09T07:13:52.199184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201678+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:13:52.201678+0000","last_peered":"2024-05-09T07:13:52.201678+0000","last_clean":"2024-05-09T07:13:52.201678+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:13:52.201678+0000","last_undegraded":"2024-05-09T07:13:52.201678+0000","last_fullsized":"2024-05-09T07:13:52.201678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201895+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:13:52.201895+0000","last_peered":"2024-05-09T07:13:52.201895+0000","last_clean":"2024-05-09T07:13:52.201895+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:13:52.201895+0000","last_undegraded":"2024-05-09T07:13:52.201895+0000","last_fullsized":"2024-05-09T07:13:52.201895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492100+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:13:51.492100+0000","last_peered":"2024-05-09T07:13:51.492100+0000","last_clean":"2024-05-09T07:13:51.492100+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:13:51.492100+0000","last_undegraded":"2024-05-09T07:13:51.492100+0000","last_fullsized":"2024-05-09T07:13:51.492100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698135+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:13:51.698135+0000","last_peered":"2024-05-09T07:13:51.698135+0000","last_clean":"2024-05-09T07:13:51.698135+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:13:51.698135+0000","last_undegraded":"2024-05-09T07:13:51.698135+0000","last_fullsized":"2024-05-09T07:13:51.698135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202333+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:13:52.202333+0000","last_peered":"2024-05-09T07:13:52.202333+0000","last_clean":"2024-05-09T07:13:52.202333+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:13:52.202333+0000","last_undegraded":"2024-05-09T07:13:52.202333+0000","last_fullsized":"2024-05-09T07:13:52.202333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491909+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:13:51.491909+0000","last_peered":"2024-05-09T07:13:51.491909+0000","last_clean":"2024-05-09T07:13:51.491909+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:13:51.491909+0000","last_undegraded":"2024-05-09T07:13:51.491909+0000","last_fullsized":"2024-05-09T07:13:51.491909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198932+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:13:52.198932+0000","last_peered":"2024-05-09T07:13:52.198932+0000","last_clean":"2024-05-09T07:13:52.198932+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:13:52.198932+0000","last_undegraded":"2024-05-09T07:13:52.198932+0000","last_fullsized":"2024-05-09T07:13:52.198932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199299+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:13:52.199299+0000","last_peered":"2024-05-09T07:13:52.199299+0000","last_clean":"2024-05-09T07:13:52.199299+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:13:52.199299+0000","last_undegraded":"2024-05-09T07:13:52.199299+0000","last_fullsized":"2024-05-09T07:13:52.199299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202849+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:13:52.202849+0000","last_peered":"2024-05-09T07:13:52.202849+0000","last_clean":"2024-05-09T07:13:52.202849+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:13:52.202849+0000","last_undegraded":"2024-05-09T07:13:52.202849+0000","last_fullsized":"2024-05-09T07:13:52.202849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698233+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:13:51.698233+0000","last_peered":"2024-05-09T07:13:51.698233+0000","last_clean":"2024-05-09T07:13:51.698233+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:13:51.698233+0000","last_undegraded":"2024-05-09T07:13:51.698233+0000","last_fullsized":"2024-05-09T07:13:51.698233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:54.466687+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:13:54.466687+0000","last_peered":"2024-05-09T07:13:54.466687+0000","last_clean":"2024-05-09T07:13:54.466687+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:13:54.466687+0000","last_undegraded":"2024-05-09T07:13:54.466687+0000","last_fullsized":"2024-05-09T07:13:54.466687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492117+0000","last_change":"2024-05-09T07:13:49.503824+0000","last_active":"2024-05-09T07:13:51.492117+0000","last_peered":"2024-05-09T07:13:51.492117+0000","last_clean":"2024-05-09T07:13:51.492117+0000","last_became_active":"2024-05-09T07:13:49.503453+0000","last_became_peered":"2024-05-09T07:13:49.503453+0000","last_unstale":"2024-05-09T07:13:51.492117+0000","last_undegraded":"2024-05-09T07:13:51.492117+0000","last_fullsized":"2024-05-09T07:13:51.492117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801449+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:13:51.801449+0000","last_peered":"2024-05-09T07:13:51.801449+0000","last_clean":"2024-05-09T07:13:51.801449+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:13:51.801449+0000","last_undegraded":"2024-05-09T07:13:51.801449+0000","last_fullsized":"2024-05-09T07:13:51.801449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490306+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:13:51.490306+0000","last_peered":"2024-05-09T07:13:51.490306+0000","last_clean":"2024-05-09T07:13:51.490306+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:13:51.490306+0000","last_undegraded":"2024-05-09T07:13:51.490306+0000","last_fullsized":"2024-05-09T07:13:51.490306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801327+0000","last_change":"2024-05-09T07:13:49.509136+0000","last_active":"2024-05-09T07:13:51.801327+0000","last_peered":"2024-05-09T07:13:51.801327+0000","last_clean":"2024-05-09T07:13:51.801327+0000","last_became_active":"2024-05-09T07:13:49.508102+0000","last_became_peered":"2024-05-09T07:13:49.508102+0000","last_unstale":"2024-05-09T07:13:51.801327+0000","last_undegraded":"2024-05-09T07:13:51.801327+0000","last_fullsized":"2024-05-09T07:13:51.801327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199289+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:13:52.199289+0000","last_peered":"2024-05-09T07:13:52.199289+0000","last_clean":"2024-05-09T07:13:52.199289+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:13:52.199289+0000","last_undegraded":"2024-05-09T07:13:52.199289+0000","last_fullsized":"2024-05-09T07:13:52.199289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202839+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:13:52.202839+0000","last_peered":"2024-05-09T07:13:52.202839+0000","last_clean":"2024-05-09T07:13:52.202839+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:13:52.202839+0000","last_undegraded":"2024-05-09T07:13:52.202839+0000","last_fullsized":"2024-05-09T07:13:52.202839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492108+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:13:51.492108+0000","last_peered":"2024-05-09T07:13:51.492108+0000","last_clean":"2024-05-09T07:13:51.492108+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:13:51.492108+0000","last_undegraded":"2024-05-09T07:13:51.492108+0000","last_fullsized":"2024-05-09T07:13:51.492108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202701+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:13:52.202701+0000","last_peered":"2024-05-09T07:13:52.202701+0000","last_clean":"2024-05-09T07:13:52.202701+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:13:52.202701+0000","last_undegraded":"2024-05-09T07:13:52.202701+0000","last_fullsized":"2024-05-09T07:13:52.202701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202142+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:13:52.202142+0000","last_peered":"2024-05-09T07:13:52.202142+0000","last_clean":"2024-05-09T07:13:52.202142+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:13:52.202142+0000","last_undegraded":"2024-05-09T07:13:52.202142+0000","last_fullsized":"2024-05-09T07:13:52.202142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491246+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:13:51.491246+0000","last_peered":"2024-05-09T07:13:51.491246+0000","last_clean":"2024-05-09T07:13:51.491246+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:13:51.491246+0000","last_undegraded":"2024-05-09T07:13:51.491246+0000","last_fullsized":"2024-05-09T07:13:51.491246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199132+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:13:52.199132+0000","last_peered":"2024-05-09T07:13:52.199132+0000","last_clean":"2024-05-09T07:13:52.199132+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:13:52.199132+0000","last_undegraded":"2024-05-09T07:13:52.199132+0000","last_fullsized":"2024-05-09T07:13:52.199132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490579+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:13:51.490579+0000","last_peered":"2024-05-09T07:13:51.490579+0000","last_clean":"2024-05-09T07:13:51.490579+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:13:51.490579+0000","last_undegraded":"2024-05-09T07:13:51.490579+0000","last_fullsized":"2024-05-09T07:13:51.490579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201450+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:13:52.201450+0000","last_peered":"2024-05-09T07:13:52.201450+0000","last_clean":"2024-05-09T07:13:52.201450+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:13:52.201450+0000","last_undegraded":"2024-05-09T07:13:52.201450+0000","last_fullsized":"2024-05-09T07:13:52.201450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202518+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:13:52.202518+0000","last_peered":"2024-05-09T07:13:52.202518+0000","last_clean":"2024-05-09T07:13:52.202518+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:13:52.202518+0000","last_undegraded":"2024-05-09T07:13:52.202518+0000","last_fullsized":"2024-05-09T07:13:52.202518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491038+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:13:51.491038+0000","last_peered":"2024-05-09T07:13:51.491038+0000","last_clean":"2024-05-09T07:13:51.491038+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:13:51.491038+0000","last_undegraded":"2024-05-09T07:13:51.491038+0000","last_fullsized":"2024-05-09T07:13:51.491038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491161+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:13:51.491161+0000","last_peered":"2024-05-09T07:13:51.491161+0000","last_clean":"2024-05-09T07:13:51.491161+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:13:51.491161+0000","last_undegraded":"2024-05-09T07:13:51.491161+0000","last_fullsized":"2024-05-09T07:13:51.491161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201962+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:13:52.201962+0000","last_peered":"2024-05-09T07:13:52.201962+0000","last_clean":"2024-05-09T07:13:52.201962+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:13:52.201962+0000","last_undegraded":"2024-05-09T07:13:52.201962+0000","last_fullsized":"2024-05-09T07:13:52.201962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199171+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:13:52.199171+0000","last_peered":"2024-05-09T07:13:52.199171+0000","last_clean":"2024-05-09T07:13:52.199171+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:13:52.199171+0000","last_undegraded":"2024-05-09T07:13:52.199171+0000","last_fullsized":"2024-05-09T07:13:52.199171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490242+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:13:51.490242+0000","last_peered":"2024-05-09T07:13:51.490242+0000","last_clean":"2024-05-09T07:13:51.490242+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:13:51.490242+0000","last_undegraded":"2024-05-09T07:13:51.490242+0000","last_fullsized":"2024-05-09T07:13:51.490242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491319+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:13:51.491319+0000","last_peered":"2024-05-09T07:13:51.491319+0000","last_clean":"2024-05-09T07:13:51.491319+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:13:51.491319+0000","last_undegraded":"2024-05-09T07:13:51.491319+0000","last_fullsized":"2024-05-09T07:13:51.491319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801024+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:13:51.801024+0000","last_peered":"2024-05-09T07:13:51.801024+0000","last_clean":"2024-05-09T07:13:51.801024+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:13:51.801024+0000","last_undegraded":"2024-05-09T07:13:51.801024+0000","last_fullsized":"2024-05-09T07:13:51.801024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492243+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:13:51.492243+0000","last_peered":"2024-05-09T07:13:51.492243+0000","last_clean":"2024-05-09T07:13:51.492243+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:13:51.492243+0000","last_undegraded":"2024-05-09T07:13:51.492243+0000","last_fullsized":"2024-05-09T07:13:51.492243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201566+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:13:52.201566+0000","last_peered":"2024-05-09T07:13:52.201566+0000","last_clean":"2024-05-09T07:13:52.201566+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:13:52.201566+0000","last_undegraded":"2024-05-09T07:13:52.201566+0000","last_fullsized":"2024-05-09T07:13:52.201566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489979+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:13:51.489979+0000","last_peered":"2024-05-09T07:13:51.489979+0000","last_clean":"2024-05-09T07:13:51.489979+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:13:51.489979+0000","last_undegraded":"2024-05-09T07:13:51.489979+0000","last_fullsized":"2024-05-09T07:13:51.489979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202280+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:13:52.202280+0000","last_peered":"2024-05-09T07:13:52.202280+0000","last_clean":"2024-05-09T07:13:52.202280+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:13:52.202280+0000","last_undegraded":"2024-05-09T07:13:52.202280+0000","last_fullsized":"2024-05-09T07:13:52.202280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199032+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:13:52.199032+0000","last_peered":"2024-05-09T07:13:52.199032+0000","last_clean":"2024-05-09T07:13:52.199032+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:13:52.199032+0000","last_undegraded":"2024-05-09T07:13:52.199032+0000","last_fullsized":"2024-05-09T07:13:52.199032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491975+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:13:51.491975+0000","last_peered":"2024-05-09T07:13:51.491975+0000","last_clean":"2024-05-09T07:13:51.491975+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:13:51.491975+0000","last_undegraded":"2024-05-09T07:13:51.491975+0000","last_fullsized":"2024-05-09T07:13:51.491975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490135+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:13:51.490135+0000","last_peered":"2024-05-09T07:13:51.490135+0000","last_clean":"2024-05-09T07:13:51.490135+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:13:51.490135+0000","last_undegraded":"2024-05-09T07:13:51.490135+0000","last_fullsized":"2024-05-09T07:13:51.490135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202331+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:13:52.202331+0000","last_peered":"2024-05-09T07:13:52.202331+0000","last_clean":"2024-05-09T07:13:52.202331+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:13:52.202331+0000","last_undegraded":"2024-05-09T07:13:52.202331+0000","last_fullsized":"2024-05-09T07:13:52.202331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492151+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:13:51.492151+0000","last_peered":"2024-05-09T07:13:51.492151+0000","last_clean":"2024-05-09T07:13:51.492151+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:13:51.492151+0000","last_undegraded":"2024-05-09T07:13:51.492151+0000","last_fullsized":"2024-05-09T07:13:51.492151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201443+0000","last_change":"2024-05-09T07:13:45.483771+0000","last_active":"2024-05-09T07:13:52.201443+0000","last_peered":"2024-05-09T07:13:52.201443+0000","last_clean":"2024-05-09T07:13:52.201443+0000","last_became_active":"2024-05-09T07:13:45.483484+0000","last_became_peered":"2024-05-09T07:13:45.483484+0000","last_unstale":"2024-05-09T07:13:52.201443+0000","last_undegraded":"2024-05-09T07:13:52.201443+0000","last_fullsized":"2024-05-09T07:13:52.201443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491076+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:13:51.491076+0000","last_peered":"2024-05-09T07:13:51.491076+0000","last_clean":"2024-05-09T07:13:51.491076+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:13:51.491076+0000","last_undegraded":"2024-05-09T07:13:51.491076+0000","last_fullsized":"2024-05-09T07:13:51.491076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202200+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:13:52.202200+0000","last_peered":"2024-05-09T07:13:52.202200+0000","last_clean":"2024-05-09T07:13:52.202200+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:13:52.202200+0000","last_undegraded":"2024-05-09T07:13:52.202200+0000","last_fullsized":"2024-05-09T07:13:52.202200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198920+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:13:52.198920+0000","last_peered":"2024-05-09T07:13:52.198920+0000","last_clean":"2024-05-09T07:13:52.198920+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:13:52.198920+0000","last_undegraded":"2024-05-09T07:13:52.198920+0000","last_fullsized":"2024-05-09T07:13:52.198920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698479+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:13:51.698479+0000","last_peered":"2024-05-09T07:13:51.698479+0000","last_clean":"2024-05-09T07:13:51.698479+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:13:51.698479+0000","last_undegraded":"2024-05-09T07:13:51.698479+0000","last_fullsized":"2024-05-09T07:13:51.698479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490136+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:13:51.490136+0000","last_peered":"2024-05-09T07:13:51.490136+0000","last_clean":"2024-05-09T07:13:51.490136+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:13:51.490136+0000","last_undegraded":"2024-05-09T07:13:51.490136+0000","last_fullsized":"2024-05-09T07:13:51.490136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202983+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:13:52.202983+0000","last_peered":"2024-05-09T07:13:52.202983+0000","last_clean":"2024-05-09T07:13:52.202983+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:13:52.202983+0000","last_undegraded":"2024-05-09T07:13:52.202983+0000","last_fullsized":"2024-05-09T07:13:52.202983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201343+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:13:52.201343+0000","last_peered":"2024-05-09T07:13:52.201343+0000","last_clean":"2024-05-09T07:13:52.201343+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:13:52.201343+0000","last_undegraded":"2024-05-09T07:13:52.201343+0000","last_fullsized":"2024-05-09T07:13:52.201343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490661+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:13:51.490661+0000","last_peered":"2024-05-09T07:13:51.490661+0000","last_clean":"2024-05-09T07:13:51.490661+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:13:51.490661+0000","last_undegraded":"2024-05-09T07:13:51.490661+0000","last_fullsized":"2024-05-09T07:13:51.490661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198914+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:13:52.198914+0000","last_peered":"2024-05-09T07:13:52.198914+0000","last_clean":"2024-05-09T07:13:52.198914+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:13:52.198914+0000","last_undegraded":"2024-05-09T07:13:52.198914+0000","last_fullsized":"2024-05-09T07:13:52.198914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491230+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:13:51.491230+0000","last_peered":"2024-05-09T07:13:51.491230+0000","last_clean":"2024-05-09T07:13:51.491230+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:13:51.491230+0000","last_undegraded":"2024-05-09T07:13:51.491230+0000","last_fullsized":"2024-05-09T07:13:51.491230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801266+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:13:51.801266+0000","last_peered":"2024-05-09T07:13:51.801266+0000","last_clean":"2024-05-09T07:13:51.801266+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:13:51.801266+0000","last_undegraded":"2024-05-09T07:13:51.801266+0000","last_fullsized":"2024-05-09T07:13:51.801266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202141+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:13:52.202141+0000","last_peered":"2024-05-09T07:13:52.202141+0000","last_clean":"2024-05-09T07:13:52.202141+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:13:52.202141+0000","last_undegraded":"2024-05-09T07:13:52.202141+0000","last_fullsized":"2024-05-09T07:13:52.202141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202227+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:13:52.202227+0000","last_peered":"2024-05-09T07:13:52.202227+0000","last_clean":"2024-05-09T07:13:52.202227+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:13:52.202227+0000","last_undegraded":"2024-05-09T07:13:52.202227+0000","last_fullsized":"2024-05-09T07:13:52.202227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698631+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:13:51.698631+0000","last_peered":"2024-05-09T07:13:51.698631+0000","last_clean":"2024-05-09T07:13:51.698631+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:13:51.698631+0000","last_undegraded":"2024-05-09T07:13:51.698631+0000","last_fullsized":"2024-05-09T07:13:51.698631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490881+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:13:51.490881+0000","last_peered":"2024-05-09T07:13:51.490881+0000","last_clean":"2024-05-09T07:13:51.490881+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:13:51.490881+0000","last_undegraded":"2024-05-09T07:13:51.490881+0000","last_fullsized":"2024-05-09T07:13:51.490881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698046+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:13:51.698046+0000","last_peered":"2024-05-09T07:13:51.698046+0000","last_clean":"2024-05-09T07:13:51.698046+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:13:51.698046+0000","last_undegraded":"2024-05-09T07:13:51.698046+0000","last_fullsized":"2024-05-09T07:13:51.698046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202156+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:13:52.202156+0000","last_peered":"2024-05-09T07:13:52.202156+0000","last_clean":"2024-05-09T07:13:52.202156+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:13:52.202156+0000","last_undegraded":"2024-05-09T07:13:52.202156+0000","last_fullsized":"2024-05-09T07:13:52.202156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698092+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:13:51.698092+0000","last_peered":"2024-05-09T07:13:51.698092+0000","last_clean":"2024-05-09T07:13:51.698092+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:13:51.698092+0000","last_undegraded":"2024-05-09T07:13:51.698092+0000","last_fullsized":"2024-05-09T07:13:51.698092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202561+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:13:52.202561+0000","last_peered":"2024-05-09T07:13:52.202561+0000","last_clean":"2024-05-09T07:13:52.202561+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:13:52.202561+0000","last_undegraded":"2024-05-09T07:13:52.202561+0000","last_fullsized":"2024-05-09T07:13:52.202561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199052+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:13:52.199052+0000","last_peered":"2024-05-09T07:13:52.199052+0000","last_clean":"2024-05-09T07:13:52.199052+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:13:52.199052+0000","last_undegraded":"2024-05-09T07:13:52.199052+0000","last_fullsized":"2024-05-09T07:13:52.199052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491163+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:13:51.491163+0000","last_peered":"2024-05-09T07:13:51.491163+0000","last_clean":"2024-05-09T07:13:51.491163+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:13:51.491163+0000","last_undegraded":"2024-05-09T07:13:51.491163+0000","last_fullsized":"2024-05-09T07:13:51.491163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698522+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:13:51.698522+0000","last_peered":"2024-05-09T07:13:51.698522+0000","last_clean":"2024-05-09T07:13:51.698522+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:13:51.698522+0000","last_undegraded":"2024-05-09T07:13:51.698522+0000","last_fullsized":"2024-05-09T07:13:51.698522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:14:02.500 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:14:02.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.632+0000 7f64a6316640 1 Processor -- start 2024-05-09T07:14:02.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.632+0000 7f64a6316640 1 -- start start 2024-05-09T07:14:02.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f64a6316640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f64a0153300 0x7f64a00c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f64a6316640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64a00bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f64a6316640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64a01563b0 con 0x7f64a00a5fb0 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f64a6316640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f64a00c1630 con 0x7f64a0153300 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f64a6316640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64a00c19d0 con 0x7f64a00c5e20 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649f7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f64a00a5fb0 0x7f64a00bbe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36294/0 (socket says 172.21.15.160:36294) 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649f7fe640 1 -- 172.21.15.160:0/3329106648 learned_addr learned my addr 172.21.15.160:0/3329106648 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649ffff640 1 --2- 172.21.15.160:0/3329106648 >> v2:172.21.15.188:3300/0 conn(0x7f64a0153300 0x7f64a00c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f64a48b3640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64a00bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 480379537 0 0) 0x7f64a01563b0 con 0x7f64a00a5fb0 2024-05-09T07:14:02.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6488003520 con 0x7f64a00a5fb0 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2872909831 0 0) 0x7f6488003520 con 0x7f64a00a5fb0 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 msgr2=0x7f64a00bc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64a00bc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 >> v2:172.21.15.188:3300/0 conn(0x7f64a0153300 msgr2=0x7f64a00c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 --2- 172.21.15.160:0/3329106648 >> v2:172.21.15.188:3300/0 conn(0x7f64a0153300 0x7f64a00c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a0156690 con 0x7f64a00a5fb0 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f6494001670 con 0x7f64a00a5fb0 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 >> v1:172.21.15.160:6789/0 conn(0x7f64a00a5fb0 legacy=0x7f64a00bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64880039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6488003f30 0x7f6488004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488004d40 0x7f64880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00c1630 con 0x7f6488004d40 2024-05-09T07:14:02.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00c19d0 con 0x7f6488003f30 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.633+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f648800b9c0 con 0x7f64a00c5e20 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488004d40 0x7f64880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f649f7fe640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6488003f30 0x7f6488004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f649ffff640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64880039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 -- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 msgr2=0x7f64880039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64880039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 -- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6488003f30 msgr2=0x7f6488004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6488003f30 0x7f6488004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 -- 172.21.15.160:0/3329106648 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6488003520 con 0x7f6488004d40 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f649ffff640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f64a00c5e20 0x7f64880039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:02.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f649f7fe640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6488003f30 0x7f6488004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.634+0000 7f64a48b3640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488004d40 0x7f64880051c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6498009f90 tx=0x7f6488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef5f49b2f455d65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498009c10 con 0x7f6488004d40 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498036070 con 0x7f6488004d40 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f649effd640 1 -- 172.21.15.160:0/3329106648 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498031630 con 0x7f6488004d40 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a6316640 1 -- 172.21.15.160:0/3329106648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488004d40 msgr2=0x7f64880051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a6316640 1 --2- 172.21.15.160:0/3329106648 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488004d40 0x7f64880051c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6498009f90 tx=0x7f6488005830 comp rx=0 tx=0).stop 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a48b3640 1 -- 172.21.15.160:0/3329106648 reap_dead start 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a6316640 1 -- 172.21.15.160:0/3329106648 shutdown_connections 2024-05-09T07:14:02.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a6316640 1 -- 172.21.15.160:0/3329106648 >> 172.21.15.160:0/3329106648 conn(0x7f64a00c7580 msgr2=0x7f64a00a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a6316640 1 -- 172.21.15.160:0/3329106648 shutdown_connections 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.635+0000 7f64a6316640 1 -- 172.21.15.160:0/3329106648 wait complete. 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.636+0000 7f64a6316640 1 Processor -- start 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 -- start start 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6488004d40 0x7f64a01ec210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01ec750 0x7f64a0071e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 0x7f64a0150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00c2420 con 0x7f64a00723c0 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f64a00c1f00 con 0x7f64a01ec750 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a6316640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f64a00bd220 con 0x7f6488004d40 2024-05-09T07:14:02.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a48b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 0x7f64a0150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a48b3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 0x7f64a0150830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43916/0 (socket says 172.21.15.160:43916) 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f64a48b3640 1 -- 172.21.15.160:0/2966042301 learned_addr learned my addr 172.21.15.160:0/2966042301 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.637+0000 7f649f7fe640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01ec750 0x7f64a0071e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f649ffff640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6488004d40 0x7f64a01ec210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a48b3640 1 -- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6488004d40 msgr2=0x7f64a01ec210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a48b3640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6488004d40 0x7f64a01ec210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a48b3640 1 -- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01ec750 msgr2=0x7f64a0071e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a48b3640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01ec750 0x7f64a0071e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a48b3640 1 -- 172.21.15.160:0/2966042301 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64a01ecbb0 con 0x7f64a00723c0 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f649f7fe640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01ec750 0x7f64a0071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f649ffff640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6488004d40 0x7f64a01ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a48b3640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 0x7f64a0150830 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6498009f90 tx=0x7f6498009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.649 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498009c10 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6498031d50 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64a01ece40 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6498044650 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.638+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64a0070960 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.639+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64a006ddc0 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.640+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6498005d10 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.640+0000 7f649cff9640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f648c06be60 0x7f648c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.640+0000 7f649ffff640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f648c06be60 0x7f648c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.640+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f64980abcb0 con 0x7f64a00723c0 2024-05-09T07:14:02.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.641+0000 7f649ffff640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f648c06be60 0x7f648c06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6490009b50 tx=0x7f6490008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.645+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f64a006ddc0 con 0x7f64a00723c0 2024-05-09T07:14:02.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.813+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64a0071b60 con 0x7f648c06be60 2024-05-09T07:14:02.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.821+0000 7f649cff9640 1 -- 172.21.15.160:0/2966042301 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f64a0071b60 con 0x7f648c06be60 2024-05-09T07:14:02.826 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:02.828 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f648c06be60 msgr2=0x7f648c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f648c06be60 0x7f648c06e320 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6490009b50 tx=0x7f6490008040 comp rx=0 tx=0).stop 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 msgr2=0x7f64a0150830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 0x7f64a0150830 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6498009f90 tx=0x7f6498009750 comp rx=0 tx=0).stop 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 shutdown_connections 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f648c06be60 0x7f648c06e320 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f64a01ec750 0x7f64a0071e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.830 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64a00723c0 0x7f64a0150830 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 --2- 172.21.15.160:0/2966042301 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6488004d40 0x7f64a01ec210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 >> 172.21.15.160:0/2966042301 conn(0x7f64a00c7580 msgr2=0x7f64a00ab980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 shutdown_connections 2024-05-09T07:14:02.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.824+0000 7f64a6316640 1 -- 172.21.15.160:0/2966042301 wait complete. 2024-05-09T07:14:02.837 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-05-09T07:14:01.182022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219468,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755252,"statfs":{"total":773094113280,"available":772869378048,"internally_reserved":0,"allocated":3383296,"data_stored":2048878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001990"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199324+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:13:52.199324+0000","last_peered":"2024-05-09T07:13:52.199324+0000","last_clean":"2024-05-09T07:13:52.199324+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:13:52.199324+0000","last_undegraded":"2024-05-09T07:13:52.199324+0000","last_fullsized":"2024-05-09T07:13:52.199324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491364+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:13:51.491364+0000","last_peered":"2024-05-09T07:13:51.491364+0000","last_clean":"2024-05-09T07:13:51.491364+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:13:51.491364+0000","last_undegraded":"2024-05-09T07:13:51.491364+0000","last_fullsized":"2024-05-09T07:13:51.491364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490779+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:13:51.490779+0000","last_peered":"2024-05-09T07:13:51.490779+0000","last_clean":"2024-05-09T07:13:51.490779+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:13:51.490779+0000","last_undegraded":"2024-05-09T07:13:51.490779+0000","last_fullsized":"2024-05-09T07:13:51.490779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491562+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:13:51.491562+0000","last_peered":"2024-05-09T07:13:51.491562+0000","last_clean":"2024-05-09T07:13:51.491562+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:13:51.491562+0000","last_undegraded":"2024-05-09T07:13:51.491562+0000","last_fullsized":"2024-05-09T07:13:51.491562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199187+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:13:52.199187+0000","last_peered":"2024-05-09T07:13:52.199187+0000","last_clean":"2024-05-09T07:13:52.199187+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:13:52.199187+0000","last_undegraded":"2024-05-09T07:13:52.199187+0000","last_fullsized":"2024-05-09T07:13:52.199187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800899+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:13:51.800899+0000","last_peered":"2024-05-09T07:13:51.800899+0000","last_clean":"2024-05-09T07:13:51.800899+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:13:51.800899+0000","last_undegraded":"2024-05-09T07:13:51.800899+0000","last_fullsized":"2024-05-09T07:13:51.800899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698480+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:13:51.698480+0000","last_peered":"2024-05-09T07:13:51.698480+0000","last_clean":"2024-05-09T07:13:51.698480+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:13:51.698480+0000","last_undegraded":"2024-05-09T07:13:51.698480+0000","last_fullsized":"2024-05-09T07:13:51.698480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202587+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:13:52.202587+0000","last_peered":"2024-05-09T07:13:52.202587+0000","last_clean":"2024-05-09T07:13:52.202587+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:13:52.202587+0000","last_undegraded":"2024-05-09T07:13:52.202587+0000","last_fullsized":"2024-05-09T07:13:52.202587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801107+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:13:51.801107+0000","last_peered":"2024-05-09T07:13:51.801107+0000","last_clean":"2024-05-09T07:13:51.801107+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:13:51.801107+0000","last_undegraded":"2024-05-09T07:13:51.801107+0000","last_fullsized":"2024-05-09T07:13:51.801107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698221+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:13:51.698221+0000","last_peered":"2024-05-09T07:13:51.698221+0000","last_clean":"2024-05-09T07:13:51.698221+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:13:51.698221+0000","last_undegraded":"2024-05-09T07:13:51.698221+0000","last_fullsized":"2024-05-09T07:13:51.698221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202703+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:13:52.202703+0000","last_peered":"2024-05-09T07:13:52.202703+0000","last_clean":"2024-05-09T07:13:52.202703+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:13:52.202703+0000","last_undegraded":"2024-05-09T07:13:52.202703+0000","last_fullsized":"2024-05-09T07:13:52.202703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800950+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:13:51.800950+0000","last_peered":"2024-05-09T07:13:51.800950+0000","last_clean":"2024-05-09T07:13:51.800950+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:13:51.800950+0000","last_undegraded":"2024-05-09T07:13:51.800950+0000","last_fullsized":"2024-05-09T07:13:51.800950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201596+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:13:52.201596+0000","last_peered":"2024-05-09T07:13:52.201596+0000","last_clean":"2024-05-09T07:13:52.201596+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:13:52.201596+0000","last_undegraded":"2024-05-09T07:13:52.201596+0000","last_fullsized":"2024-05-09T07:13:52.201596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698232+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:13:51.698232+0000","last_peered":"2024-05-09T07:13:51.698232+0000","last_clean":"2024-05-09T07:13:51.698232+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:13:51.698232+0000","last_undegraded":"2024-05-09T07:13:51.698232+0000","last_fullsized":"2024-05-09T07:13:51.698232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801528+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:13:51.801528+0000","last_peered":"2024-05-09T07:13:51.801528+0000","last_clean":"2024-05-09T07:13:51.801528+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:13:51.801528+0000","last_undegraded":"2024-05-09T07:13:51.801528+0000","last_fullsized":"2024-05-09T07:13:51.801528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491149+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:13:51.491149+0000","last_peered":"2024-05-09T07:13:51.491149+0000","last_clean":"2024-05-09T07:13:51.491149+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:13:51.491149+0000","last_undegraded":"2024-05-09T07:13:51.491149+0000","last_fullsized":"2024-05-09T07:13:51.491149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491381+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:13:51.491381+0000","last_peered":"2024-05-09T07:13:51.491381+0000","last_clean":"2024-05-09T07:13:51.491381+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:13:51.491381+0000","last_undegraded":"2024-05-09T07:13:51.491381+0000","last_fullsized":"2024-05-09T07:13:51.491381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801475+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:13:51.801475+0000","last_peered":"2024-05-09T07:13:51.801475+0000","last_clean":"2024-05-09T07:13:51.801475+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:13:51.801475+0000","last_undegraded":"2024-05-09T07:13:51.801475+0000","last_fullsized":"2024-05-09T07:13:51.801475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491169+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:13:51.491169+0000","last_peered":"2024-05-09T07:13:51.491169+0000","last_clean":"2024-05-09T07:13:51.491169+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:13:51.491169+0000","last_undegraded":"2024-05-09T07:13:51.491169+0000","last_fullsized":"2024-05-09T07:13:51.491169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490742+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:13:51.490742+0000","last_peered":"2024-05-09T07:13:51.490742+0000","last_clean":"2024-05-09T07:13:51.490742+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:13:51.490742+0000","last_undegraded":"2024-05-09T07:13:51.490742+0000","last_fullsized":"2024-05-09T07:13:51.490742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491961+0000","last_change":"2024-05-09T07:13:49.938952+0000","last_active":"2024-05-09T07:13:51.491961+0000","last_peered":"2024-05-09T07:13:51.491961+0000","last_clean":"2024-05-09T07:13:51.491961+0000","last_became_active":"2024-05-09T07:13:49.938552+0000","last_became_peered":"2024-05-09T07:13:49.938552+0000","last_unstale":"2024-05-09T07:13:51.491961+0000","last_undegraded":"2024-05-09T07:13:51.491961+0000","last_fullsized":"2024-05-09T07:13:51.491961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801398+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:13:51.801398+0000","last_peered":"2024-05-09T07:13:51.801398+0000","last_clean":"2024-05-09T07:13:51.801398+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:13:51.801398+0000","last_undegraded":"2024-05-09T07:13:51.801398+0000","last_fullsized":"2024-05-09T07:13:51.801398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697811+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:13:51.697811+0000","last_peered":"2024-05-09T07:13:51.697811+0000","last_clean":"2024-05-09T07:13:51.697811+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:13:51.697811+0000","last_undegraded":"2024-05-09T07:13:51.697811+0000","last_fullsized":"2024-05-09T07:13:51.697811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801577+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:13:51.801577+0000","last_peered":"2024-05-09T07:13:51.801577+0000","last_clean":"2024-05-09T07:13:51.801577+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:13:51.801577+0000","last_undegraded":"2024-05-09T07:13:51.801577+0000","last_fullsized":"2024-05-09T07:13:51.801577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199329+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:13:52.199329+0000","last_peered":"2024-05-09T07:13:52.199329+0000","last_clean":"2024-05-09T07:13:52.199329+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:13:52.199329+0000","last_undegraded":"2024-05-09T07:13:52.199329+0000","last_fullsized":"2024-05-09T07:13:52.199329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202341+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:13:52.202341+0000","last_peered":"2024-05-09T07:13:52.202341+0000","last_clean":"2024-05-09T07:13:52.202341+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:13:52.202341+0000","last_undegraded":"2024-05-09T07:13:52.202341+0000","last_fullsized":"2024-05-09T07:13:52.202341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800919+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:13:51.800919+0000","last_peered":"2024-05-09T07:13:51.800919+0000","last_clean":"2024-05-09T07:13:51.800919+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:13:51.800919+0000","last_undegraded":"2024-05-09T07:13:51.800919+0000","last_fullsized":"2024-05-09T07:13:51.800919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492207+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:13:51.492207+0000","last_peered":"2024-05-09T07:13:51.492207+0000","last_clean":"2024-05-09T07:13:51.492207+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:13:51.492207+0000","last_undegraded":"2024-05-09T07:13:51.492207+0000","last_fullsized":"2024-05-09T07:13:51.492207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491259+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:13:51.491259+0000","last_peered":"2024-05-09T07:13:51.491259+0000","last_clean":"2024-05-09T07:13:51.491259+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:13:51.491259+0000","last_undegraded":"2024-05-09T07:13:51.491259+0000","last_fullsized":"2024-05-09T07:13:51.491259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490763+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:13:51.490763+0000","last_peered":"2024-05-09T07:13:51.490763+0000","last_clean":"2024-05-09T07:13:51.490763+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:13:51.490763+0000","last_undegraded":"2024-05-09T07:13:51.490763+0000","last_fullsized":"2024-05-09T07:13:51.490763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491690+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:13:51.491690+0000","last_peered":"2024-05-09T07:13:51.491690+0000","last_clean":"2024-05-09T07:13:51.491690+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:13:51.491690+0000","last_undegraded":"2024-05-09T07:13:51.491690+0000","last_fullsized":"2024-05-09T07:13:51.491690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698566+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:13:51.698566+0000","last_peered":"2024-05-09T07:13:51.698566+0000","last_clean":"2024-05-09T07:13:51.698566+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:13:51.698566+0000","last_undegraded":"2024-05-09T07:13:51.698566+0000","last_fullsized":"2024-05-09T07:13:51.698566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198963+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:13:52.198963+0000","last_peered":"2024-05-09T07:13:52.198963+0000","last_clean":"2024-05-09T07:13:52.198963+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:13:52.198963+0000","last_undegraded":"2024-05-09T07:13:52.198963+0000","last_fullsized":"2024-05-09T07:13:52.198963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202806+0000","last_change":"2024-05-09T07:13:45.484793+0000","last_active":"2024-05-09T07:13:52.202806+0000","last_peered":"2024-05-09T07:13:52.202806+0000","last_clean":"2024-05-09T07:13:52.202806+0000","last_became_active":"2024-05-09T07:13:45.484248+0000","last_became_peered":"2024-05-09T07:13:45.484248+0000","last_unstale":"2024-05-09T07:13:52.202806+0000","last_undegraded":"2024-05-09T07:13:52.202806+0000","last_fullsized":"2024-05-09T07:13:52.202806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491769+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:13:51.491769+0000","last_peered":"2024-05-09T07:13:51.491769+0000","last_clean":"2024-05-09T07:13:51.491769+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:13:51.491769+0000","last_undegraded":"2024-05-09T07:13:51.491769+0000","last_fullsized":"2024-05-09T07:13:51.491769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491114+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:13:51.491114+0000","last_peered":"2024-05-09T07:13:51.491114+0000","last_clean":"2024-05-09T07:13:51.491114+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:13:51.491114+0000","last_undegraded":"2024-05-09T07:13:51.491114+0000","last_fullsized":"2024-05-09T07:13:51.491114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801532+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:13:51.801532+0000","last_peered":"2024-05-09T07:13:51.801532+0000","last_clean":"2024-05-09T07:13:51.801532+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:13:51.801532+0000","last_undegraded":"2024-05-09T07:13:51.801532+0000","last_fullsized":"2024-05-09T07:13:51.801532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202356+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:13:52.202356+0000","last_peered":"2024-05-09T07:13:52.202356+0000","last_clean":"2024-05-09T07:13:52.202356+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:13:52.202356+0000","last_undegraded":"2024-05-09T07:13:52.202356+0000","last_fullsized":"2024-05-09T07:13:52.202356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491646+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:13:51.491646+0000","last_peered":"2024-05-09T07:13:51.491646+0000","last_clean":"2024-05-09T07:13:51.491646+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:13:51.491646+0000","last_undegraded":"2024-05-09T07:13:51.491646+0000","last_fullsized":"2024-05-09T07:13:51.491646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490640+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:13:51.490640+0000","last_peered":"2024-05-09T07:13:51.490640+0000","last_clean":"2024-05-09T07:13:51.490640+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:13:51.490640+0000","last_undegraded":"2024-05-09T07:13:51.490640+0000","last_fullsized":"2024-05-09T07:13:51.490640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490075+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:13:51.490075+0000","last_peered":"2024-05-09T07:13:51.490075+0000","last_clean":"2024-05-09T07:13:51.490075+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:13:51.490075+0000","last_undegraded":"2024-05-09T07:13:51.490075+0000","last_fullsized":"2024-05-09T07:13:51.490075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492028+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:13:51.492028+0000","last_peered":"2024-05-09T07:13:51.492028+0000","last_clean":"2024-05-09T07:13:51.492028+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:13:51.492028+0000","last_undegraded":"2024-05-09T07:13:51.492028+0000","last_fullsized":"2024-05-09T07:13:51.492028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198795+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:13:52.198795+0000","last_peered":"2024-05-09T07:13:52.198795+0000","last_clean":"2024-05-09T07:13:52.198795+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:13:52.198795+0000","last_undegraded":"2024-05-09T07:13:52.198795+0000","last_fullsized":"2024-05-09T07:13:52.198795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697830+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:13:51.697830+0000","last_peered":"2024-05-09T07:13:51.697830+0000","last_clean":"2024-05-09T07:13:51.697830+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:13:51.697830+0000","last_undegraded":"2024-05-09T07:13:51.697830+0000","last_fullsized":"2024-05-09T07:13:51.697830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199305+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:13:52.199305+0000","last_peered":"2024-05-09T07:13:52.199305+0000","last_clean":"2024-05-09T07:13:52.199305+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:13:52.199305+0000","last_undegraded":"2024-05-09T07:13:52.199305+0000","last_fullsized":"2024-05-09T07:13:52.199305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491484+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:13:51.491484+0000","last_peered":"2024-05-09T07:13:51.491484+0000","last_clean":"2024-05-09T07:13:51.491484+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:13:51.491484+0000","last_undegraded":"2024-05-09T07:13:51.491484+0000","last_fullsized":"2024-05-09T07:13:51.491484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202232+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:13:52.202232+0000","last_peered":"2024-05-09T07:13:52.202232+0000","last_clean":"2024-05-09T07:13:52.202232+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:13:52.202232+0000","last_undegraded":"2024-05-09T07:13:52.202232+0000","last_fullsized":"2024-05-09T07:13:52.202232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697989+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:13:51.697989+0000","last_peered":"2024-05-09T07:13:51.697989+0000","last_clean":"2024-05-09T07:13:51.697989+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:13:51.697989+0000","last_undegraded":"2024-05-09T07:13:51.697989+0000","last_fullsized":"2024-05-09T07:13:51.697989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698344+0000","last_change":"2024-05-09T07:13:45.490251+0000","last_active":"2024-05-09T07:13:51.698344+0000","last_peered":"2024-05-09T07:13:51.698344+0000","last_clean":"2024-05-09T07:13:51.698344+0000","last_became_active":"2024-05-09T07:13:45.489707+0000","last_became_peered":"2024-05-09T07:13:45.489707+0000","last_unstale":"2024-05-09T07:13:51.698344+0000","last_undegraded":"2024-05-09T07:13:51.698344+0000","last_fullsized":"2024-05-09T07:13:51.698344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490413+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:13:51.490413+0000","last_peered":"2024-05-09T07:13:51.490413+0000","last_clean":"2024-05-09T07:13:51.490413+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:13:51.490413+0000","last_undegraded":"2024-05-09T07:13:51.490413+0000","last_fullsized":"2024-05-09T07:13:51.490413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201756+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:13:52.201756+0000","last_peered":"2024-05-09T07:13:52.201756+0000","last_clean":"2024-05-09T07:13:52.201756+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:13:52.201756+0000","last_undegraded":"2024-05-09T07:13:52.201756+0000","last_fullsized":"2024-05-09T07:13:52.201756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202663+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:13:52.202663+0000","last_peered":"2024-05-09T07:13:52.202663+0000","last_clean":"2024-05-09T07:13:52.202663+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:13:52.202663+0000","last_undegraded":"2024-05-09T07:13:52.202663+0000","last_fullsized":"2024-05-09T07:13:52.202663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490395+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:13:51.490395+0000","last_peered":"2024-05-09T07:13:51.490395+0000","last_clean":"2024-05-09T07:13:51.490395+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:13:51.490395+0000","last_undegraded":"2024-05-09T07:13:51.490395+0000","last_fullsized":"2024-05-09T07:13:51.490395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199283+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:13:52.199283+0000","last_peered":"2024-05-09T07:13:52.199283+0000","last_clean":"2024-05-09T07:13:52.199283+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:13:52.199283+0000","last_undegraded":"2024-05-09T07:13:52.199283+0000","last_fullsized":"2024-05-09T07:13:52.199283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698652+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:13:51.698652+0000","last_peered":"2024-05-09T07:13:51.698652+0000","last_clean":"2024-05-09T07:13:51.698652+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:13:51.698652+0000","last_undegraded":"2024-05-09T07:13:51.698652+0000","last_fullsized":"2024-05-09T07:13:51.698652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490223+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:13:51.490223+0000","last_peered":"2024-05-09T07:13:51.490223+0000","last_clean":"2024-05-09T07:13:51.490223+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:13:51.490223+0000","last_undegraded":"2024-05-09T07:13:51.490223+0000","last_fullsized":"2024-05-09T07:13:51.490223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201384+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:13:52.201384+0000","last_peered":"2024-05-09T07:13:52.201384+0000","last_clean":"2024-05-09T07:13:52.201384+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:13:52.201384+0000","last_undegraded":"2024-05-09T07:13:52.201384+0000","last_fullsized":"2024-05-09T07:13:52.201384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698489+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:13:51.698489+0000","last_peered":"2024-05-09T07:13:51.698489+0000","last_clean":"2024-05-09T07:13:51.698489+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:13:51.698489+0000","last_undegraded":"2024-05-09T07:13:51.698489+0000","last_fullsized":"2024-05-09T07:13:51.698489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801555+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:13:51.801555+0000","last_peered":"2024-05-09T07:13:51.801555+0000","last_clean":"2024-05-09T07:13:51.801555+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:13:51.801555+0000","last_undegraded":"2024-05-09T07:13:51.801555+0000","last_fullsized":"2024-05-09T07:13:51.801555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800811+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:13:51.800811+0000","last_peered":"2024-05-09T07:13:51.800811+0000","last_clean":"2024-05-09T07:13:51.800811+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:13:51.800811+0000","last_undegraded":"2024-05-09T07:13:51.800811+0000","last_fullsized":"2024-05-09T07:13:51.800811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698054+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:13:51.698054+0000","last_peered":"2024-05-09T07:13:51.698054+0000","last_clean":"2024-05-09T07:13:51.698054+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:13:51.698054+0000","last_undegraded":"2024-05-09T07:13:51.698054+0000","last_fullsized":"2024-05-09T07:13:51.698054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489838+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:13:51.489838+0000","last_peered":"2024-05-09T07:13:51.489838+0000","last_clean":"2024-05-09T07:13:51.489838+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:13:51.489838+0000","last_undegraded":"2024-05-09T07:13:51.489838+0000","last_fullsized":"2024-05-09T07:13:51.489838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199184+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:13:52.199184+0000","last_peered":"2024-05-09T07:13:52.199184+0000","last_clean":"2024-05-09T07:13:52.199184+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:13:52.199184+0000","last_undegraded":"2024-05-09T07:13:52.199184+0000","last_fullsized":"2024-05-09T07:13:52.199184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201678+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:13:52.201678+0000","last_peered":"2024-05-09T07:13:52.201678+0000","last_clean":"2024-05-09T07:13:52.201678+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:13:52.201678+0000","last_undegraded":"2024-05-09T07:13:52.201678+0000","last_fullsized":"2024-05-09T07:13:52.201678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201895+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:13:52.201895+0000","last_peered":"2024-05-09T07:13:52.201895+0000","last_clean":"2024-05-09T07:13:52.201895+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:13:52.201895+0000","last_undegraded":"2024-05-09T07:13:52.201895+0000","last_fullsized":"2024-05-09T07:13:52.201895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492100+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:13:51.492100+0000","last_peered":"2024-05-09T07:13:51.492100+0000","last_clean":"2024-05-09T07:13:51.492100+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:13:51.492100+0000","last_undegraded":"2024-05-09T07:13:51.492100+0000","last_fullsized":"2024-05-09T07:13:51.492100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698135+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:13:51.698135+0000","last_peered":"2024-05-09T07:13:51.698135+0000","last_clean":"2024-05-09T07:13:51.698135+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:13:51.698135+0000","last_undegraded":"2024-05-09T07:13:51.698135+0000","last_fullsized":"2024-05-09T07:13:51.698135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202333+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:13:52.202333+0000","last_peered":"2024-05-09T07:13:52.202333+0000","last_clean":"2024-05-09T07:13:52.202333+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:13:52.202333+0000","last_undegraded":"2024-05-09T07:13:52.202333+0000","last_fullsized":"2024-05-09T07:13:52.202333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491909+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:13:51.491909+0000","last_peered":"2024-05-09T07:13:51.491909+0000","last_clean":"2024-05-09T07:13:51.491909+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:13:51.491909+0000","last_undegraded":"2024-05-09T07:13:51.491909+0000","last_fullsized":"2024-05-09T07:13:51.491909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198932+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:13:52.198932+0000","last_peered":"2024-05-09T07:13:52.198932+0000","last_clean":"2024-05-09T07:13:52.198932+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:13:52.198932+0000","last_undegraded":"2024-05-09T07:13:52.198932+0000","last_fullsized":"2024-05-09T07:13:52.198932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199299+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:13:52.199299+0000","last_peered":"2024-05-09T07:13:52.199299+0000","last_clean":"2024-05-09T07:13:52.199299+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:13:52.199299+0000","last_undegraded":"2024-05-09T07:13:52.199299+0000","last_fullsized":"2024-05-09T07:13:52.199299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202849+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:13:52.202849+0000","last_peered":"2024-05-09T07:13:52.202849+0000","last_clean":"2024-05-09T07:13:52.202849+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:13:52.202849+0000","last_undegraded":"2024-05-09T07:13:52.202849+0000","last_fullsized":"2024-05-09T07:13:52.202849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698233+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:13:51.698233+0000","last_peered":"2024-05-09T07:13:51.698233+0000","last_clean":"2024-05-09T07:13:51.698233+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:13:51.698233+0000","last_undegraded":"2024-05-09T07:13:51.698233+0000","last_fullsized":"2024-05-09T07:13:51.698233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'8","reported_seq":32,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:54.466687+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:13:54.466687+0000","last_peered":"2024-05-09T07:13:54.466687+0000","last_clean":"2024-05-09T07:13:54.466687+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:13:54.466687+0000","last_undegraded":"2024-05-09T07:13:54.466687+0000","last_fullsized":"2024-05-09T07:13:54.466687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492117+0000","last_change":"2024-05-09T07:13:49.503824+0000","last_active":"2024-05-09T07:13:51.492117+0000","last_peered":"2024-05-09T07:13:51.492117+0000","last_clean":"2024-05-09T07:13:51.492117+0000","last_became_active":"2024-05-09T07:13:49.503453+0000","last_became_peered":"2024-05-09T07:13:49.503453+0000","last_unstale":"2024-05-09T07:13:51.492117+0000","last_undegraded":"2024-05-09T07:13:51.492117+0000","last_fullsized":"2024-05-09T07:13:51.492117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801449+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:13:51.801449+0000","last_peered":"2024-05-09T07:13:51.801449+0000","last_clean":"2024-05-09T07:13:51.801449+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:13:51.801449+0000","last_undegraded":"2024-05-09T07:13:51.801449+0000","last_fullsized":"2024-05-09T07:13:51.801449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490306+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:13:51.490306+0000","last_peered":"2024-05-09T07:13:51.490306+0000","last_clean":"2024-05-09T07:13:51.490306+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:13:51.490306+0000","last_undegraded":"2024-05-09T07:13:51.490306+0000","last_fullsized":"2024-05-09T07:13:51.490306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801327+0000","last_change":"2024-05-09T07:13:49.509136+0000","last_active":"2024-05-09T07:13:51.801327+0000","last_peered":"2024-05-09T07:13:51.801327+0000","last_clean":"2024-05-09T07:13:51.801327+0000","last_became_active":"2024-05-09T07:13:49.508102+0000","last_became_peered":"2024-05-09T07:13:49.508102+0000","last_unstale":"2024-05-09T07:13:51.801327+0000","last_undegraded":"2024-05-09T07:13:51.801327+0000","last_fullsized":"2024-05-09T07:13:51.801327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199289+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:13:52.199289+0000","last_peered":"2024-05-09T07:13:52.199289+0000","last_clean":"2024-05-09T07:13:52.199289+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:13:52.199289+0000","last_undegraded":"2024-05-09T07:13:52.199289+0000","last_fullsized":"2024-05-09T07:13:52.199289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202839+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:13:52.202839+0000","last_peered":"2024-05-09T07:13:52.202839+0000","last_clean":"2024-05-09T07:13:52.202839+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:13:52.202839+0000","last_undegraded":"2024-05-09T07:13:52.202839+0000","last_fullsized":"2024-05-09T07:13:52.202839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492108+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:13:51.492108+0000","last_peered":"2024-05-09T07:13:51.492108+0000","last_clean":"2024-05-09T07:13:51.492108+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:13:51.492108+0000","last_undegraded":"2024-05-09T07:13:51.492108+0000","last_fullsized":"2024-05-09T07:13:51.492108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202701+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:13:52.202701+0000","last_peered":"2024-05-09T07:13:52.202701+0000","last_clean":"2024-05-09T07:13:52.202701+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:13:52.202701+0000","last_undegraded":"2024-05-09T07:13:52.202701+0000","last_fullsized":"2024-05-09T07:13:52.202701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202142+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:13:52.202142+0000","last_peered":"2024-05-09T07:13:52.202142+0000","last_clean":"2024-05-09T07:13:52.202142+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:13:52.202142+0000","last_undegraded":"2024-05-09T07:13:52.202142+0000","last_fullsized":"2024-05-09T07:13:52.202142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491246+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:13:51.491246+0000","last_peered":"2024-05-09T07:13:51.491246+0000","last_clean":"2024-05-09T07:13:51.491246+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:13:51.491246+0000","last_undegraded":"2024-05-09T07:13:51.491246+0000","last_fullsized":"2024-05-09T07:13:51.491246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199132+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:13:52.199132+0000","last_peered":"2024-05-09T07:13:52.199132+0000","last_clean":"2024-05-09T07:13:52.199132+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:13:52.199132+0000","last_undegraded":"2024-05-09T07:13:52.199132+0000","last_fullsized":"2024-05-09T07:13:52.199132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490579+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:13:51.490579+0000","last_peered":"2024-05-09T07:13:51.490579+0000","last_clean":"2024-05-09T07:13:51.490579+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:13:51.490579+0000","last_undegraded":"2024-05-09T07:13:51.490579+0000","last_fullsized":"2024-05-09T07:13:51.490579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201450+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:13:52.201450+0000","last_peered":"2024-05-09T07:13:52.201450+0000","last_clean":"2024-05-09T07:13:52.201450+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:13:52.201450+0000","last_undegraded":"2024-05-09T07:13:52.201450+0000","last_fullsized":"2024-05-09T07:13:52.201450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202518+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:13:52.202518+0000","last_peered":"2024-05-09T07:13:52.202518+0000","last_clean":"2024-05-09T07:13:52.202518+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:13:52.202518+0000","last_undegraded":"2024-05-09T07:13:52.202518+0000","last_fullsized":"2024-05-09T07:13:52.202518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491038+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:13:51.491038+0000","last_peered":"2024-05-09T07:13:51.491038+0000","last_clean":"2024-05-09T07:13:51.491038+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:13:51.491038+0000","last_undegraded":"2024-05-09T07:13:51.491038+0000","last_fullsized":"2024-05-09T07:13:51.491038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491161+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:13:51.491161+0000","last_peered":"2024-05-09T07:13:51.491161+0000","last_clean":"2024-05-09T07:13:51.491161+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:13:51.491161+0000","last_undegraded":"2024-05-09T07:13:51.491161+0000","last_fullsized":"2024-05-09T07:13:51.491161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201962+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:13:52.201962+0000","last_peered":"2024-05-09T07:13:52.201962+0000","last_clean":"2024-05-09T07:13:52.201962+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:13:52.201962+0000","last_undegraded":"2024-05-09T07:13:52.201962+0000","last_fullsized":"2024-05-09T07:13:52.201962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199171+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:13:52.199171+0000","last_peered":"2024-05-09T07:13:52.199171+0000","last_clean":"2024-05-09T07:13:52.199171+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:13:52.199171+0000","last_undegraded":"2024-05-09T07:13:52.199171+0000","last_fullsized":"2024-05-09T07:13:52.199171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490242+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:13:51.490242+0000","last_peered":"2024-05-09T07:13:51.490242+0000","last_clean":"2024-05-09T07:13:51.490242+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:13:51.490242+0000","last_undegraded":"2024-05-09T07:13:51.490242+0000","last_fullsized":"2024-05-09T07:13:51.490242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491319+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:13:51.491319+0000","last_peered":"2024-05-09T07:13:51.491319+0000","last_clean":"2024-05-09T07:13:51.491319+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:13:51.491319+0000","last_undegraded":"2024-05-09T07:13:51.491319+0000","last_fullsized":"2024-05-09T07:13:51.491319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801024+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:13:51.801024+0000","last_peered":"2024-05-09T07:13:51.801024+0000","last_clean":"2024-05-09T07:13:51.801024+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:13:51.801024+0000","last_undegraded":"2024-05-09T07:13:51.801024+0000","last_fullsized":"2024-05-09T07:13:51.801024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492243+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:13:51.492243+0000","last_peered":"2024-05-09T07:13:51.492243+0000","last_clean":"2024-05-09T07:13:51.492243+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:13:51.492243+0000","last_undegraded":"2024-05-09T07:13:51.492243+0000","last_fullsized":"2024-05-09T07:13:51.492243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201566+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:13:52.201566+0000","last_peered":"2024-05-09T07:13:52.201566+0000","last_clean":"2024-05-09T07:13:52.201566+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:13:52.201566+0000","last_undegraded":"2024-05-09T07:13:52.201566+0000","last_fullsized":"2024-05-09T07:13:52.201566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489979+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:13:51.489979+0000","last_peered":"2024-05-09T07:13:51.489979+0000","last_clean":"2024-05-09T07:13:51.489979+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:13:51.489979+0000","last_undegraded":"2024-05-09T07:13:51.489979+0000","last_fullsized":"2024-05-09T07:13:51.489979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202280+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:13:52.202280+0000","last_peered":"2024-05-09T07:13:52.202280+0000","last_clean":"2024-05-09T07:13:52.202280+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:13:52.202280+0000","last_undegraded":"2024-05-09T07:13:52.202280+0000","last_fullsized":"2024-05-09T07:13:52.202280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199032+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:13:52.199032+0000","last_peered":"2024-05-09T07:13:52.199032+0000","last_clean":"2024-05-09T07:13:52.199032+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:13:52.199032+0000","last_undegraded":"2024-05-09T07:13:52.199032+0000","last_fullsized":"2024-05-09T07:13:52.199032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491975+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:13:51.491975+0000","last_peered":"2024-05-09T07:13:51.491975+0000","last_clean":"2024-05-09T07:13:51.491975+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:13:51.491975+0000","last_undegraded":"2024-05-09T07:13:51.491975+0000","last_fullsized":"2024-05-09T07:13:51.491975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490135+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:13:51.490135+0000","last_peered":"2024-05-09T07:13:51.490135+0000","last_clean":"2024-05-09T07:13:51.490135+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:13:51.490135+0000","last_undegraded":"2024-05-09T07:13:51.490135+0000","last_fullsized":"2024-05-09T07:13:51.490135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202331+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:13:52.202331+0000","last_peered":"2024-05-09T07:13:52.202331+0000","last_clean":"2024-05-09T07:13:52.202331+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:13:52.202331+0000","last_undegraded":"2024-05-09T07:13:52.202331+0000","last_fullsized":"2024-05-09T07:13:52.202331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492151+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:13:51.492151+0000","last_peered":"2024-05-09T07:13:51.492151+0000","last_clean":"2024-05-09T07:13:51.492151+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:13:51.492151+0000","last_undegraded":"2024-05-09T07:13:51.492151+0000","last_fullsized":"2024-05-09T07:13:51.492151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201443+0000","last_change":"2024-05-09T07:13:45.483771+0000","last_active":"2024-05-09T07:13:52.201443+0000","last_peered":"2024-05-09T07:13:52.201443+0000","last_clean":"2024-05-09T07:13:52.201443+0000","last_became_active":"2024-05-09T07:13:45.483484+0000","last_became_peered":"2024-05-09T07:13:45.483484+0000","last_unstale":"2024-05-09T07:13:52.201443+0000","last_undegraded":"2024-05-09T07:13:52.201443+0000","last_fullsized":"2024-05-09T07:13:52.201443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491076+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:13:51.491076+0000","last_peered":"2024-05-09T07:13:51.491076+0000","last_clean":"2024-05-09T07:13:51.491076+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:13:51.491076+0000","last_undegraded":"2024-05-09T07:13:51.491076+0000","last_fullsized":"2024-05-09T07:13:51.491076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202200+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:13:52.202200+0000","last_peered":"2024-05-09T07:13:52.202200+0000","last_clean":"2024-05-09T07:13:52.202200+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:13:52.202200+0000","last_undegraded":"2024-05-09T07:13:52.202200+0000","last_fullsized":"2024-05-09T07:13:52.202200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198920+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:13:52.198920+0000","last_peered":"2024-05-09T07:13:52.198920+0000","last_clean":"2024-05-09T07:13:52.198920+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:13:52.198920+0000","last_undegraded":"2024-05-09T07:13:52.198920+0000","last_fullsized":"2024-05-09T07:13:52.198920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698479+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:13:51.698479+0000","last_peered":"2024-05-09T07:13:51.698479+0000","last_clean":"2024-05-09T07:13:51.698479+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:13:51.698479+0000","last_undegraded":"2024-05-09T07:13:51.698479+0000","last_fullsized":"2024-05-09T07:13:51.698479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490136+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:13:51.490136+0000","last_peered":"2024-05-09T07:13:51.490136+0000","last_clean":"2024-05-09T07:13:51.490136+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:13:51.490136+0000","last_undegraded":"2024-05-09T07:13:51.490136+0000","last_fullsized":"2024-05-09T07:13:51.490136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202983+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:13:52.202983+0000","last_peered":"2024-05-09T07:13:52.202983+0000","last_clean":"2024-05-09T07:13:52.202983+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:13:52.202983+0000","last_undegraded":"2024-05-09T07:13:52.202983+0000","last_fullsized":"2024-05-09T07:13:52.202983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201343+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:13:52.201343+0000","last_peered":"2024-05-09T07:13:52.201343+0000","last_clean":"2024-05-09T07:13:52.201343+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:13:52.201343+0000","last_undegraded":"2024-05-09T07:13:52.201343+0000","last_fullsized":"2024-05-09T07:13:52.201343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490661+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:13:51.490661+0000","last_peered":"2024-05-09T07:13:51.490661+0000","last_clean":"2024-05-09T07:13:51.490661+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:13:51.490661+0000","last_undegraded":"2024-05-09T07:13:51.490661+0000","last_fullsized":"2024-05-09T07:13:51.490661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198914+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:13:52.198914+0000","last_peered":"2024-05-09T07:13:52.198914+0000","last_clean":"2024-05-09T07:13:52.198914+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:13:52.198914+0000","last_undegraded":"2024-05-09T07:13:52.198914+0000","last_fullsized":"2024-05-09T07:13:52.198914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491230+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:13:51.491230+0000","last_peered":"2024-05-09T07:13:51.491230+0000","last_clean":"2024-05-09T07:13:51.491230+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:13:51.491230+0000","last_undegraded":"2024-05-09T07:13:51.491230+0000","last_fullsized":"2024-05-09T07:13:51.491230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801266+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:13:51.801266+0000","last_peered":"2024-05-09T07:13:51.801266+0000","last_clean":"2024-05-09T07:13:51.801266+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:13:51.801266+0000","last_undegraded":"2024-05-09T07:13:51.801266+0000","last_fullsized":"2024-05-09T07:13:51.801266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202141+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:13:52.202141+0000","last_peered":"2024-05-09T07:13:52.202141+0000","last_clean":"2024-05-09T07:13:52.202141+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:13:52.202141+0000","last_undegraded":"2024-05-09T07:13:52.202141+0000","last_fullsized":"2024-05-09T07:13:52.202141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202227+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:13:52.202227+0000","last_peered":"2024-05-09T07:13:52.202227+0000","last_clean":"2024-05-09T07:13:52.202227+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:13:52.202227+0000","last_undegraded":"2024-05-09T07:13:52.202227+0000","last_fullsized":"2024-05-09T07:13:52.202227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698631+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:13:51.698631+0000","last_peered":"2024-05-09T07:13:51.698631+0000","last_clean":"2024-05-09T07:13:51.698631+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:13:51.698631+0000","last_undegraded":"2024-05-09T07:13:51.698631+0000","last_fullsized":"2024-05-09T07:13:51.698631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490881+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:13:51.490881+0000","last_peered":"2024-05-09T07:13:51.490881+0000","last_clean":"2024-05-09T07:13:51.490881+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:13:51.490881+0000","last_undegraded":"2024-05-09T07:13:51.490881+0000","last_fullsized":"2024-05-09T07:13:51.490881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698046+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:13:51.698046+0000","last_peered":"2024-05-09T07:13:51.698046+0000","last_clean":"2024-05-09T07:13:51.698046+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:13:51.698046+0000","last_undegraded":"2024-05-09T07:13:51.698046+0000","last_fullsized":"2024-05-09T07:13:51.698046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202156+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:13:52.202156+0000","last_peered":"2024-05-09T07:13:52.202156+0000","last_clean":"2024-05-09T07:13:52.202156+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:13:52.202156+0000","last_undegraded":"2024-05-09T07:13:52.202156+0000","last_fullsized":"2024-05-09T07:13:52.202156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698092+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:13:51.698092+0000","last_peered":"2024-05-09T07:13:51.698092+0000","last_clean":"2024-05-09T07:13:51.698092+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:13:51.698092+0000","last_undegraded":"2024-05-09T07:13:51.698092+0000","last_fullsized":"2024-05-09T07:13:51.698092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202561+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:13:52.202561+0000","last_peered":"2024-05-09T07:13:52.202561+0000","last_clean":"2024-05-09T07:13:52.202561+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:13:52.202561+0000","last_undegraded":"2024-05-09T07:13:52.202561+0000","last_fullsized":"2024-05-09T07:13:52.202561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199052+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:13:52.199052+0000","last_peered":"2024-05-09T07:13:52.199052+0000","last_clean":"2024-05-09T07:13:52.199052+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:13:52.199052+0000","last_undegraded":"2024-05-09T07:13:52.199052+0000","last_fullsized":"2024-05-09T07:13:52.199052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491163+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:13:51.491163+0000","last_peered":"2024-05-09T07:13:51.491163+0000","last_clean":"2024-05-09T07:13:51.491163+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:13:51.491163+0000","last_undegraded":"2024-05-09T07:13:51.491163+0000","last_fullsized":"2024-05-09T07:13:51.491163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698522+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:13:51.698522+0000","last_peered":"2024-05-09T07:13:51.698522+0000","last_clean":"2024-05-09T07:13:51.698522+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:13:51.698522+0000","last_undegraded":"2024-05-09T07:13:51.698522+0000","last_fullsized":"2024-05-09T07:13:51.698522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:14:02.844 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-05-09T07:14:02.844 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T07:14:02.844 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-09T07:14:02.845 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-09T07:14:02.845 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-09T07:14:02.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 Processor -- start 2024-05-09T07:14:02.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 -- start start 2024-05-09T07:14:02.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e78153300 0x7f7e780c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780a5fb0 0x7f7e780bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7e781563b0 con 0x7f7e780c5e20 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f7e780c1630 con 0x7f7e78153300 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.976+0000 7f7e7e934640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7e780c19d0 con 0x7f7e780a5fb0 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e77fff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f7e78153300 0x7f7e780c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e777fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780a5fb0 0x7f7e780bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e7ced1640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f7e780c5e20 0x7f7e780bc360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43196/0 (socket says 172.21.15.160:43196) 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e7ced1640 1 -- 172.21.15.160:0/594389571 learned_addr learned my addr 172.21.15.160:0/594389571 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e777fe640 1 -- 172.21.15.160:0/594389571 >> v2:172.21.15.188:3300/0 conn(0x7f7e78153300 msgr2=0x7f7e780c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e777fe640 1 --2- 172.21.15.160:0/594389571 >> v2:172.21.15.188:3300/0 conn(0x7f7e78153300 0x7f7e780c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e777fe640 1 -- 172.21.15.160:0/594389571 >> v1:172.21.15.188:6789/0 conn(0x7f7e780c5e20 legacy=0x7f7e780bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.977+0000 7f7e777fe640 1 -- 172.21.15.160:0/594389571 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e78156690 con 0x7f7e780a5fb0 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e777fe640 1 --2- 172.21.15.160:0/594389571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780a5fb0 0x7f7e780bbe20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e600097a0 tx=0x7f7e6002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69622ea0b43f503c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e76ffd640 1 -- 172.21.15.160:0/594389571 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60037070 con 0x7f7e780a5fb0 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e76ffd640 1 -- 172.21.15.160:0/594389571 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e60032440 con 0x7f7e780a5fb0 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e76ffd640 1 -- 172.21.15.160:0/594389571 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e60032810 con 0x7f7e780a5fb0 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e7e934640 1 -- 172.21.15.160:0/594389571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780a5fb0 msgr2=0x7f7e780bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e7e934640 1 --2- 172.21.15.160:0/594389571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780a5fb0 0x7f7e780bbe20 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e600097a0 tx=0x7f7e6002a180 comp rx=0 tx=0).stop 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e7e934640 1 -- 172.21.15.160:0/594389571 shutdown_connections 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e7e934640 1 --2- 172.21.15.160:0/594389571 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780a5fb0 0x7f7e780bbe20 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e7e934640 1 --2- 172.21.15.160:0/594389571 >> v2:172.21.15.188:3300/0 conn(0x7f7e78153300 0x7f7e780c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.978+0000 7f7e7e934640 1 -- 172.21.15.160:0/594389571 >> 172.21.15.160:0/594389571 conn(0x7f7e780c7580 msgr2=0x7f7e780a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.979+0000 7f7e7e934640 1 -- 172.21.15.160:0/594389571 shutdown_connections 2024-05-09T07:14:02.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.979+0000 7f7e7e934640 1 -- 172.21.15.160:0/594389571 wait complete. 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 Processor -- start 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 -- start start 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e780a5fb0 0x7f7e781ec130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780c5e20 0x7f7e781ec670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 0x7f7e781de440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7e780c2420 con 0x7f7e78153300 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.980+0000 7f7e7e934640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7e780c1f00 con 0x7f7e780a5fb0 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7e934640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7e780bd220 con 0x7f7e780c5e20 2024-05-09T07:14:02.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 0x7f7e781de440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 0x7f7e781de440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43930/0 (socket says 172.21.15.160:43930) 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 -- 172.21.15.160:0/1034987036 learned_addr learned my addr 172.21.15.160:0/1034987036 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e777fe640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780c5e20 0x7f7e781ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e77fff640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e780a5fb0 0x7f7e781ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 -- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780c5e20 msgr2=0x7f7e781ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780c5e20 0x7f7e781ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 -- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e780a5fb0 msgr2=0x7f7e781ec130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e780a5fb0 0x7f7e781ec130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 -- 172.21.15.160:0/1034987036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e781ecbb0 con 0x7f7e78153300 2024-05-09T07:14:02.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e77fff640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e780a5fb0 0x7f7e781ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7ced1640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 0x7f7e781de440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000caa0 tx=0x7f7e7000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70017070 con 0x7f7e78153300 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7e7000ac00 con 0x7f7e78153300 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e781519b0 con 0x7f7e78153300 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e70012400 con 0x7f7e78153300 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.981+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e78152470 con 0x7f7e78153300 2024-05-09T07:14:02.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.982+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e780756c0 con 0x7f7e78153300 2024-05-09T07:14:02.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.983+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7e7001a070 con 0x7f7e78153300 2024-05-09T07:14:02.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.983+0000 7f7e74ff9640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7e5806be30 0x7f7e5806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:02.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.983+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7e7008aca0 con 0x7f7e78153300 2024-05-09T07:14:02.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.983+0000 7f7e77fff640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7e5806be30 0x7f7e5806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:02.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.984+0000 7f7e77fff640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7e5806be30 0x7f7e5806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009f30 tx=0x7f7e68009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:02.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:02.988+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7e70058d50 con 0x7f7e78153300 2024-05-09T07:14:03.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.209+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f7e780bdab0 con 0x7f7e78153300 2024-05-09T07:14:03.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.212+0000 7f7e74ff9640 1 -- 172.21.15.160:0/1034987036 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+172946 (secure 0 0 0) 0x7f7e7005d050 con 0x7f7e78153300 2024-05-09T07:14:03.216 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:03.217 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.214+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7e5806be30 msgr2=0x7f7e5806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.214+0000 7f7e7e934640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7e5806be30 0x7f7e5806e2f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f7e68009f30 tx=0x7f7e68009190 comp rx=0 tx=0).stop 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.214+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 msgr2=0x7f7e781de440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.214+0000 7f7e7e934640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 0x7f7e781de440 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7e7000caa0 tx=0x7f7e7000cf60 comp rx=0 tx=0).stop 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 shutdown_connections 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7e5806be30 0x7f7e5806e2f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7e78153300 0x7f7e781de440 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7e780c5e20 0x7f7e781ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 --2- 172.21.15.160:0/1034987036 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7e780a5fb0 0x7f7e781ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 >> 172.21.15.160:0/1034987036 conn(0x7f7e780c7580 msgr2=0x7f7e780c5630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 shutdown_connections 2024-05-09T07:14:03.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.215+0000 7f7e7e934640 1 -- 172.21.15.160:0/1034987036 wait complete. 2024-05-09T07:14:03.227 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":5,"flags":0,"active_gid":4099,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2408879908},{"type":"v1","addr":"172.21.15.160:6833","nonce":2408879908}]},"active_addr":"172.21.15.160:6833/2408879908","active_change":"2024-05-09T07:13:25.169638+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":4001893830}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":1987614303}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2321507264}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.160:0","nonce":2792018705}]}]} 2024-05-09T07:14:03.232 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-09T07:14:03.232 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-09T07:14:03.232 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:14:03.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.332+0000 7f9a10d4f640 1 Processor -- start 2024-05-09T07:14:03.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.332+0000 7f9a10d4f640 1 -- start start 2024-05-09T07:14:03.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.333+0000 7f9a10d4f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a0c0c36e0 0x7f9a0c06d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.333+0000 7f9a10d4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c06d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.333+0000 7f9a10d4f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9a0c0c8af0 con 0x7f9a0c1595c0 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.333+0000 7f9a10d4f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9a0c0c2240 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.333+0000 7f9a10d4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c1599a0 con 0x7f9a0c0c8550 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a09d9b640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9a0c0c36e0 0x7f9a0c06d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a0ad9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c06d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a0a59c640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f9a0c1595c0 0x7f9a0c0c45b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43202/0 (socket says 172.21.15.160:43202) 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a0a59c640 1 -- 172.21.15.160:0/1811817532 learned_addr learned my addr 172.21.15.160:0/1811817532 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a09d9b640 1 -- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 msgr2=0x7f9a0c06d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a09d9b640 1 --2- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c06d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a09d9b640 1 -- 172.21.15.160:0/1811817532 >> v1:172.21.15.188:6789/0 conn(0x7f9a0c1595c0 legacy=0x7f9a0c0c45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a0959a640 1 -- 172.21.15.160:0/1811817532 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 527391896 0 0) 0x7f9a0c0c8af0 con 0x7f9a0c1595c0 2024-05-09T07:14:03.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.334+0000 7f9a09d9b640 1 -- 172.21.15.160:0/1811817532 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c0c24e0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.335+0000 7f9a09d9b640 1 --2- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c06d130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99f40097a0 tx=0x7f99f402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39a98d07ee2e9ac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.335+0000 7f9a0959a640 1 -- 172.21.15.160:0/1811817532 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4037070 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.335+0000 7f9a0959a640 1 -- 172.21.15.160:0/1811817532 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f402adb0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.335+0000 7f9a0959a640 1 -- 172.21.15.160:0/1811817532 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4032660 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.335+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 msgr2=0x7f9a0c06d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.335+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c06d130 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99f40097a0 tx=0x7f99f402a060 comp rx=0 tx=0).stop 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.336+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1811817532 shutdown_connections 2024-05-09T07:14:03.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.336+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c06d130 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.336+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1811817532 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c06d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.336+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1811817532 >> 172.21.15.160:0/1811817532 conn(0x7f9a0c0aaa90 msgr2=0x7f9a0c0aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:03.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.336+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1811817532 shutdown_connections 2024-05-09T07:14:03.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.336+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1811817532 wait complete. 2024-05-09T07:14:03.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.337+0000 7f9a10d4f640 1 Processor -- start 2024-05-09T07:14:03.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.337+0000 7f9a10d4f640 1 -- start start 2024-05-09T07:14:03.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a10d4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a10d4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a10d4f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 0x7f9a0c14fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a10d4f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c0c3310 con 0x7f9a0c1595c0 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a10d4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c0c2aa0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a10d4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9a0c0c4cc0 con 0x7f9a0c0c8550 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 0x7f9a0c14fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a09d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53342/0 (socket says 172.21.15.160:53342) 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 0x7f9a0c14fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43932/0 (socket says 172.21.15.160:43932) 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 -- 172.21.15.160:0/1056214056 learned_addr learned my addr 172.21.15.160:0/1056214056 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 -- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 msgr2=0x7f9a0c151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 -- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 msgr2=0x7f9a0c14fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 0x7f9a0c14fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 -- 172.21.15.160:0/1056214056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c14d0b0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0ad9d640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 0x7f9a0c14fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a0a59c640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c153720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c890 tx=0x7f99fc00cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.338+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc018070 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.339+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc00a640 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.339+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c0702b0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.339+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc00aa30 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.339+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c0adc60 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.343 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.339+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c0756c0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.340+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f99fc00abd0 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.340+0000 7f9a02ffd640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f99ec06be30 0x7f99ec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.341+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f99fc08a780 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.341+0000 7f9a09d9b640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f99ec06be30 0x7f99ec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.344 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.341+0000 7f9a09d9b640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f99ec06be30 0x7f99ec06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f99f4034000 tx=0x7f99f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:03.347 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.346+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f99fc058720 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.519+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9a0c0c3080 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.521+0000 7f9a02ffd640 1 -- 172.21.15.160:0/1056214056 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f99fc05ca20 con 0x7f9a0c0c36e0 2024-05-09T07:14:03.523 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:03.523 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f99ec06be30 msgr2=0x7f99ec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f99ec06be30 0x7f99ec06e2f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f99f4034000 tx=0x7f99f402e000 comp rx=0 tx=0).stop 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 msgr2=0x7f9a0c153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c153720 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00c890 tx=0x7f99fc00cd50 comp rx=0 tx=0).stop 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 shutdown_connections 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f99ec06be30 0x7f99ec06e2f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9a0c1595c0 0x7f9a0c14fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9a0c0c8550 0x7f9a0c151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.525 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 --2- 172.21.15.160:0/1056214056 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9a0c0c36e0 0x7f9a0c153720 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 >> 172.21.15.160:0/1056214056 conn(0x7f9a0c0aaa90 msgr2=0x7f9a0c0a8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:03.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 shutdown_connections 2024-05-09T07:14:03.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.523+0000 7f9a10d4f640 1 -- 172.21.15.160:0/1056214056 wait complete. 2024-05-09T07:14:03.537 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-09T07:14:03.537 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:14:03.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.635+0000 7f69a81e3640 1 Processor -- start 2024-05-09T07:14:03.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a81e3640 1 -- start start 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a81e3640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f69a00c5d80 0x7f69a006b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a81e3640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a00a5e50 con 0x7f69a0153fb0 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a81e3640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69a01534d0 con 0x7f69a00c2f00 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a81e3640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f69a0153b20 con 0x7f69a00c5d80 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a6780640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f69a0153fb0 0x7f69a015dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36300/0 (socket says 172.21.15.160:36300) 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a6780640 1 -- 172.21.15.160:0/2718830947 learned_addr learned my addr 172.21.15.160:0/2718830947 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:03.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.636+0000 7f69a577e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f69a00c5d80 0x7f69a006b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.637+0000 7f69a577e640 1 -- 172.21.15.160:0/2718830947 >> v1:172.21.15.188:6789/0 conn(0x7f69a00c2f00 legacy=0x7f69a01568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.637+0000 7f69a577e640 1 -- 172.21.15.160:0/2718830947 >> v1:172.21.15.160:6789/0 conn(0x7f69a0153fb0 legacy=0x7f69a015dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.637+0000 7f69a577e640 1 -- 172.21.15.160:0/2718830947 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a01537b0 con 0x7f69a00c5d80 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.637+0000 7f69a4f7d640 1 -- 172.21.15.160:0/2718830947 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1992456797 0 0) 0x7f69a00a5e50 con 0x7f69a0153fb0 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.637+0000 7f69a577e640 1 --2- 172.21.15.160:0/2718830947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c5d80 0x7f69a006b160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f699c0097a0 tx=0x7f699c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19702321bc64e993 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a4f7d640 1 -- 172.21.15.160:0/2718830947 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c037070 con 0x7f69a00c5d80 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a4f7d640 1 -- 172.21.15.160:0/2718830947 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f699c02adc0 con 0x7f69a00c5d80 2024-05-09T07:14:03.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a4f7d640 1 -- 172.21.15.160:0/2718830947 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699c032660 con 0x7f69a00c5d80 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 -- 172.21.15.160:0/2718830947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c5d80 msgr2=0x7f69a006b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 --2- 172.21.15.160:0/2718830947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c5d80 0x7f69a006b160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f699c0097a0 tx=0x7f699c02a0b0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 -- 172.21.15.160:0/2718830947 shutdown_connections 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 --2- 172.21.15.160:0/2718830947 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c5d80 0x7f69a006b160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 -- 172.21.15.160:0/2718830947 >> 172.21.15.160:0/2718830947 conn(0x7f69a00c7580 msgr2=0x7f69a00a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 -- 172.21.15.160:0/2718830947 shutdown_connections 2024-05-09T07:14:03.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.638+0000 7f69a81e3640 1 -- 172.21.15.160:0/2718830947 wait complete. 2024-05-09T07:14:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.639+0000 7f69a81e3640 1 Processor -- start 2024-05-09T07:14:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.639+0000 7f69a81e3640 1 -- start start 2024-05-09T07:14:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a81e3640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 0x7f69a0073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a81e3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a00c5d80 0x7f69a00aa960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a81e3640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a0153fb0 0x7f69a0071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a81e3640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f69a00c6730 con 0x7f69a00c2f00 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a81e3640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f69a00c65b0 con 0x7f69a0153fb0 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a81e3640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f69a00bb8f0 con 0x7f69a00c5d80 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 0x7f69a0073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 0x7f69a0073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43958/0 (socket says 172.21.15.160:43958) 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 -- 172.21.15.160:0/1358054298 learned_addr learned my addr 172.21.15.160:0/1358054298 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a577e640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a00c5d80 0x7f69a00aa960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a6780640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a0153fb0 0x7f69a0071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 -- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a00c5d80 msgr2=0x7f69a00aa960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a00c5d80 0x7f69a00aa960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 -- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a0153fb0 msgr2=0x7f69a0071240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a0153fb0 0x7f69a0071240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.640+0000 7f69a5f7f640 1 -- 172.21.15.160:0/1358054298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a01dd4b0 con 0x7f69a00c2f00 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.641+0000 7f69a5f7f640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 0x7f69a0073140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f699400b420 tx=0x7f699400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:03.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.641+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994011830 con 0x7f69a00c2f00 2024-05-09T07:14:03.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.641+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6994016070 con 0x7f69a00c2f00 2024-05-09T07:14:03.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.641+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994011c10 con 0x7f69a00c2f00 2024-05-09T07:14:03.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.641+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a01dd790 con 0x7f69a00c2f00 2024-05-09T07:14:03.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.641+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a01ddb90 con 0x7f69a00c2f00 2024-05-09T07:14:03.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.642+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a01dd960 con 0x7f69a00c2f00 2024-05-09T07:14:03.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.642+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f699401b460 con 0x7f69a00c2f00 2024-05-09T07:14:03.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.643+0000 7f69927fc640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f698406be30 0x7f698406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:03.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.643+0000 7f69a577e640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f698406be30 0x7f698406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:03.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.643+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f699408ad40 con 0x7f69a00c2f00 2024-05-09T07:14:03.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.643+0000 7f69a577e640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f698406be30 0x7f698406e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f699c034000 tx=0x7f699c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:03.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.647+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6994020020 con 0x7f69a00c2f00 2024-05-09T07:14:03.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.820+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69a00756c0 con 0x7f69a00c2f00 2024-05-09T07:14:03.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.821+0000 7f69927fc640 1 -- 172.21.15.160:0/1358054298 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f6994059320 con 0x7f69a00c2f00 2024-05-09T07:14:03.823 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:03.823 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:14:03.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f698406be30 msgr2=0x7f698406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f698406be30 0x7f698406e2f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f699c034000 tx=0x7f699c02e000 comp rx=0 tx=0).stop 2024-05-09T07:14:03.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 msgr2=0x7f69a0073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:03.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 0x7f69a0073140 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f699400b420 tx=0x7f699400b8e0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 shutdown_connections 2024-05-09T07:14:03.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f698406be30 0x7f698406e2f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f69a0153fb0 0x7f69a0071240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f69a00c5d80 0x7f69a00aa960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 --2- 172.21.15.160:0/1358054298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69a00c2f00 0x7f69a0073140 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:03.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 >> 172.21.15.160:0/1358054298 conn(0x7f69a00c7580 msgr2=0x7f69a00c5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:03.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 shutdown_connections 2024-05-09T07:14:03.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:03.823+0000 7f69a81e3640 1 -- 172.21.15.160:0/1358054298 wait complete. 2024-05-09T07:14:03.836 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-09T07:14:03.837 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-09T07:14:03.837 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-09T07:14:03.837 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-09T07:14:03.838 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-09T07:14:03.838 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-09T07:14:03.839 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-09T07:14:03.839 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-09T07:14:04.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.011+0000 7f03052ba640 1 Processor -- start 2024-05-09T07:14:04.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.012+0000 7ff09670e640 1 Processor -- start 2024-05-09T07:14:04.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.011+0000 7f03052ba640 1 -- start start 2024-05-09T07:14:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7ff09670e640 1 -- start start 2024-05-09T07:14:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f03052ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f03000c7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f03052ba640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03000734d0 0x7f030006e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f03052ba640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03000c8040 con 0x7f030006edb0 2024-05-09T07:14:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f03052ba640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f03000c7250 con 0x7f03000734d0 2024-05-09T07:14:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f03052ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f03000c6c00 con 0x7f0300156e30 2024-05-09T07:14:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f02fe7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03000734d0 0x7f030006e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f02feffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f03000c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f02fe7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f03000734d0 0x7f030006e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:43972/0 (socket says 172.21.15.160:43972) 2024-05-09T07:14:04.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f02feffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f03000c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44260/0 (socket says 172.21.15.160:44260) 2024-05-09T07:14:04.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.013+0000 7f02feffd640 1 -- 172.21.15.160:0/3198971751 learned_addr learned my addr 172.21.15.160:0/3198971751 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7f02fdffb640 1 -- 172.21.15.160:0/3198971751 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1688157236 0 0) 0x7f03000c8040 con 0x7f030006edb0 2024-05-09T07:14:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7f02fdffb640 1 -- 172.21.15.160:0/3198971751 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f02e8003520 con 0x7f030006edb0 2024-05-09T07:14:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff09670e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ff0880d4870 0x7ff08804a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.018 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff09670e640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff08804b380 con 0x7ff0880d7ab0 2024-05-09T07:14:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff09670e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0880d72f0 con 0x7ff0880073c0 2024-05-09T07:14:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff09670e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ff0880d7620 con 0x7ff0880d4870 2024-05-09T07:14:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff094f0b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7ff0880073c0 0x7ff088046150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43212/0 (socket says 172.21.15.160:43212) 2024-05-09T07:14:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff094f0b640 1 -- 172.21.15.160:0/3046866396 learned_addr learned my addr 172.21.15.160:0/3046866396 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:04.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7ff09570c640 1 --2- 172.21.15.160:0/3046866396 >> v2:172.21.15.160:3300/0 conn(0x7ff0880d4870 0x7ff08804a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7f02fe7fc640 1 -- 172.21.15.160:0/3198971751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 msgr2=0x7f03000c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7f02fe7fc640 1 --2- 172.21.15.160:0/3198971751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f03000c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7f02fe7fc640 1 -- 172.21.15.160:0/3198971751 >> v1:172.21.15.160:6789/0 conn(0x7f030006edb0 legacy=0x7f0300159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7f02fe7fc640 1 -- 172.21.15.160:0/3198971751 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03000c6ee0 con 0x7f03000734d0 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 341102638 0 0) 0x7ff0880d72f0 con 0x7ff0880073c0 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff07c005730 con 0x7ff0880073c0 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3036516427 0 0) 0x7ff08804b380 con 0x7ff0880d7ab0 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff0880d72f0 con 0x7ff0880d7ab0 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1842665838 0 0) 0x7ff07c005730 con 0x7ff0880073c0 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 >> v2:172.21.15.160:3300/0 conn(0x7ff0880d4870 msgr2=0x7ff08804a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 --2- 172.21.15.160:0/3046866396 >> v2:172.21.15.160:3300/0 conn(0x7ff0880d4870 0x7ff08804a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 >> v1:172.21.15.160:6789/0 conn(0x7ff0880d7ab0 legacy=0x7ff08800b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0880096f0 con 0x7ff0880073c0 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7ff09004fe40 con 0x7ff0880073c0 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 >> v1:172.21.15.188:6789/0 conn(0x7ff0880073c0 legacy=0x7ff088046150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7ff087fff640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07c005d50 0x7ff07c006130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff087fff640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07c006a10 0x7ff07c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff094f0b640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07c006a10 0x7ff07c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7f3de3268640 1 Processor -- start 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7f3de3268640 1 -- start start 2024-05-09T07:14:04.022 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff09570c640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07c005d50 0x7ff07c006130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7fcf058d4640 1 Processor -- start 2024-05-09T07:14:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.014+0000 7fcf058d4640 1 -- start start 2024-05-09T07:14:04.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff087fff640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c0073d0 0x7ff07c0147c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcf058d4640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcef80f9450 0x7fcef80f5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880d7620 con 0x7ff07c006a10 2024-05-09T07:14:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c00bbc0 con 0x7ff07c0073d0 2024-05-09T07:14:04.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff07c00bdc0 con 0x7ff07c005d50 2024-05-09T07:14:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7ff095f0d640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c0073d0 0x7ff07c0147c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7f02fe7fc640 1 --2- 172.21.15.160:0/3198971751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03000734d0 0x7f030006e870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f02f40097a0 tx=0x7f02f402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d467108923444cf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f02fdffb640 1 -- 172.21.15.160:0/3198971751 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f4037070 con 0x7f03000734d0 2024-05-09T07:14:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f02fdffb640 1 -- 172.21.15.160:0/3198971751 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f4032440 con 0x7f03000734d0 2024-05-09T07:14:04.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f02fdffb640 1 -- 172.21.15.160:0/3198971751 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f402ae60 con 0x7f03000734d0 2024-05-09T07:14:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcf058d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80f56a0 0x7fcef81028e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcf058d4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcef800c3d0 con 0x7fcef80463f0 2024-05-09T07:14:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcf058d4640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcef80f8ee0 con 0x7fcef80f9450 2024-05-09T07:14:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f3de3268640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3ddc0c4780 0x7f3ddc0c4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f3de3268640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ddc06d280 con 0x7f3ddc156d80 2024-05-09T07:14:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f3de3268640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ddc06d560 con 0x7f3ddc06d9e0 2024-05-09T07:14:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7f3de3268640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3ddc0c2a80 con 0x7f3ddc0c4780 2024-05-09T07:14:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcf058d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcef8006b10 con 0x7fcef80f56a0 2024-05-09T07:14:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f3de1a65640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f3ddc06d9e0 0x7f3ddc0c15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43236/0 (socket says 172.21.15.160:43236) 2024-05-09T07:14:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f3de1a65640 1 -- 172.21.15.160:0/2498535576 learned_addr learned my addr 172.21.15.160:0/2498535576 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:04.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcefe7fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcef80f9450 0x7fcef80f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcefe7fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcef80f9450 0x7fcef80f5160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53360/0 (socket says 172.21.15.160:53360) 2024-05-09T07:14:04.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fcefe7fc640 1 -- 172.21.15.160:0/3765049476 learned_addr learned my addr 172.21.15.160:0/3765049476 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1303592294 0 0) 0x7f3ddc06d560 con 0x7f3ddc06d9e0 2024-05-09T07:14:04.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3dc8003520 con 0x7f3ddc06d9e0 2024-05-09T07:14:04.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7ff094f0b640 1 -- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07c005d50 msgr2=0x7ff07c006130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7ff094f0b640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff07c005d50 0x7ff07c006130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7ff094f0b640 1 -- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c0073d0 msgr2=0x7ff07c0147c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7ff094f0b640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c0073d0 0x7ff07c0147c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7ff094f0b640 1 -- 172.21.15.160:0/3046866396 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c005730 con 0x7ff07c006a10 2024-05-09T07:14:04.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7ff095f0d640 1 -- 172.21.15.160:0/3046866396 reap_dead start 2024-05-09T07:14:04.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f03052ba640 1 -- 172.21.15.160:0/3198971751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03000734d0 msgr2=0x7f030006e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f03052ba640 1 --2- 172.21.15.160:0/3198971751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03000734d0 0x7f030006e870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f02f40097a0 tx=0x7f02f402a110 comp rx=0 tx=0).stop 2024-05-09T07:14:04.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7ff094f0b640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07c006a10 0x7ff07c006e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff09006a0b0 tx=0x7ff07c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d16fa214aa3e216 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7f3de2a67640 1 --2- 172.21.15.160:0/2498535576 >> v2:172.21.15.160:3300/0 conn(0x7f3ddc0c4780 0x7f3ddc0c4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 42395356 0 0) 0x7f3dc8003520 con 0x7f3ddc06d9e0 2024-05-09T07:14:04.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 >> v2:172.21.15.160:3300/0 conn(0x7f3ddc0c4780 msgr2=0x7f3ddc0c4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.015+0000 7fceff7fe640 1 --2- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80f56a0 0x7fcef81028e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7fcefe7fc640 1 -- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80f56a0 msgr2=0x7fcef81028e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7fcefe7fc640 1 --2- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80f56a0 0x7fcef81028e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7fcefe7fc640 1 -- 172.21.15.160:0/3765049476 >> v1:172.21.15.188:6789/0 conn(0x7fcef80463f0 legacy=0x7fcef80f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.016+0000 7fcefe7fc640 1 -- 172.21.15.160:0/3765049476 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8006df0 con 0x7fcef80f9450 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7fcefe7fc640 1 --2- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f9450 0x7fcef80f5160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf00069bf0 tx=0x7fcf0007a7f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=493791d819b30223 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.017+0000 7fcefdffb640 1 -- 172.21.15.160:0/3765049476 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0009d8c0 con 0x7fcef80f9450 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7fcefdffb640 1 -- 172.21.15.160:0/3765049476 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf0009da60 con 0x7fcef80f9450 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.018+0000 7fcefdffb640 1 -- 172.21.15.160:0/3765049476 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0009de10 con 0x7fcef80f9450 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 --2- 172.21.15.160:0/2498535576 >> v2:172.21.15.160:3300/0 conn(0x7f3ddc0c4780 0x7f3ddc0c4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 >> v1:172.21.15.160:6789/0 conn(0x7f3ddc156d80 legacy=0x7f3ddc15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc0c2d60 con 0x7f3ddc06d9e0 2024-05-09T07:14:04.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f03052ba640 1 -- 172.21.15.160:0/3198971751 shutdown_connections 2024-05-09T07:14:04.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f03052ba640 1 --2- 172.21.15.160:0/3198971751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f03000734d0 0x7f030006e870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f03052ba640 1 --2- 172.21.15.160:0/3198971751 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f03000c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f03052ba640 1 -- 172.21.15.160:0/3198971751 >> 172.21.15.160:0/3198971751 conn(0x7f030006fb10 msgr2=0x7f030006ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.033 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f3dcc001670 con 0x7f3ddc06d9e0 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 >> v1:172.21.15.188:6789/0 conn(0x7f3ddc06d9e0 legacy=0x7f3ddc0c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3dc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc8005230 0x7f3dc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc0c2a80 con 0x7f3dc80039b0 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3dc800c190 con 0x7f3dc8005230 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3dc800c390 con 0x7f3dc8004800 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de2a67640 1 --2- 172.21.15.160:0/2498535576 >> v2:172.21.15.160:3300/0 conn(0x7f3ddc0c4780 0x7f3ddc0c4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de1a65640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3dc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff090069150 con 0x7ff07c006a10 2024-05-09T07:14:04.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 -- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f9450 msgr2=0x7fcef80f5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7f3de2266640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7ff087fff640 1 -- 172.21.15.160:0/3046866396 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0900aa070 con 0x7ff07c006a10 2024-05-09T07:14:04.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de2a67640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc8005230 0x7f3dc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f119ffff640 1 Processor -- start 2024-05-09T07:14:04.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 --2- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f9450 0x7fcef80f5160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf00069bf0 tx=0x7fcf0007a7f0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 -- 172.21.15.160:0/3765049476 shutdown_connections 2024-05-09T07:14:04.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 --2- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f9450 0x7fcef80f5160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 --2- 172.21.15.160:0/3765049476 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80f56a0 0x7fcef81028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 -- 172.21.15.160:0/3765049476 >> 172.21.15.160:0/3765049476 conn(0x7fcef8005ec0 msgr2=0x7fcef80062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 -- 172.21.15.160:0/3765049476 shutdown_connections 2024-05-09T07:14:04.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.019+0000 7fcf058d4640 1 -- 172.21.15.160:0/3765049476 wait complete. 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de1a65640 1 -- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc8005230 msgr2=0x7f3dc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de1a65640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc8005230 0x7f3dc800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de1a65640 1 -- 172.21.15.160:0/2498535576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc80039b0 msgr2=0x7f3dc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de1a65640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3dc80039b0 0x7f3dc8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de1a65640 1 -- 172.21.15.160:0/2498535576 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dc8003520 con 0x7f3dc8004800 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de2a67640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3dc8005230 0x7f3dc800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f3de2a67640 1 -- 172.21.15.160:0/2498535576 reap_dead start 2024-05-09T07:14:04.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f03052ba640 1 -- 172.21.15.160:0/3198971751 shutdown_connections 2024-05-09T07:14:04.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7ff09670e640 1 -- 172.21.15.160:0/3046866396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07c006a10 msgr2=0x7ff07c006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7ff09670e640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07c006a10 0x7ff07c006e90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff09006a0b0 tx=0x7ff07c007a40 comp rx=0 tx=0).stop 2024-05-09T07:14:04.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f03052ba640 1 -- 172.21.15.160:0/3198971751 wait complete. 2024-05-09T07:14:04.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7ff09670e640 1 -- 172.21.15.160:0/3046866396 shutdown_connections 2024-05-09T07:14:04.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f3de1a65640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3dc8004c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c110 tx=0x7f3dc8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d45fa509d51d3d39 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7ff09670e640 1 --2- 172.21.15.160:0/3046866396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff07c006a10 0x7ff07c006e90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7ff09670e640 1 -- 172.21.15.160:0/3046866396 >> 172.21.15.160:0/3046866396 conn(0x7ff088005e90 msgr2=0x7ff0880062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7fcf058d4640 1 Processor -- start 2024-05-09T07:14:04.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007c40 con 0x7f3dc8004800 2024-05-09T07:14:04.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dcc005ad0 con 0x7f3dc8004800 2024-05-09T07:14:04.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f3de1264640 1 -- 172.21.15.160:0/2498535576 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dcc007070 con 0x7f3dc8004800 2024-05-09T07:14:04.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.020+0000 7f119ffff640 1 -- start start 2024-05-09T07:14:04.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.023+0000 7f03052ba640 1 Processor -- start 2024-05-09T07:14:04.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f5c7d653640 1 Processor -- start 2024-05-09T07:14:04.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f119ffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f11a00c3680 0x7f11a0159590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f119ffff640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a015cc20 con 0x7f11a0070f20 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f119ffff640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f11a015cf00 con 0x7f11a0156b30 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f119ffff640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f11a015d830 con 0x7f11a00c3680 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7f119e7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f11a00c3680 0x7f11a0159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f119e7fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f11a00c3680 0x7f11a0159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44016/0 (socket says 172.21.15.160:44016) 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f119e7fc640 1 -- 172.21.15.160:0/652156649 learned_addr learned my addr 172.21.15.160:0/652156649 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f119e7fc640 1 -- 172.21.15.160:0/652156649 >> v1:172.21.15.188:6789/0 conn(0x7f11a0156b30 legacy=0x7f11a006ef90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f119e7fc640 1 -- 172.21.15.160:0/652156649 >> v1:172.21.15.160:6789/0 conn(0x7f11a0070f20 legacy=0x7f11a0159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f5c7d653640 1 -- start start 2024-05-09T07:14:04.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7f119e7fc640 1 -- 172.21.15.160:0/652156649 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a015d1e0 con 0x7f11a00c3680 2024-05-09T07:14:04.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7fcf058d4640 1 -- start start 2024-05-09T07:14:04.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.023+0000 7f3de3268640 1 -- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 msgr2=0x7f3dc8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.023+0000 7f3de3268640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3dc8004c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc00c110 tx=0x7f3dc8005830 comp rx=0 tx=0).stop 2024-05-09T07:14:04.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcf058d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80463f0 0x7fcef8183fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.023+0000 7f03052ba640 1 -- start start 2024-05-09T07:14:04.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f3de3268640 1 -- 172.21.15.160:0/2498535576 shutdown_connections 2024-05-09T07:14:04.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f3de3268640 1 --2- 172.21.15.160:0/2498535576 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3dc8004c80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f3de3268640 1 -- 172.21.15.160:0/2498535576 >> 172.21.15.160:0/2498535576 conn(0x7f3ddc0a85a0 msgr2=0x7f3ddc0a89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcf058d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f56a0 0x7fcef8184500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcf058d4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef80f9450 0x7fcef8191830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcf058d4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcef80f7f60 con 0x7fcef80f9450 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcf058d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcef80f82b0 con 0x7fcef80f56a0 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcf058d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcef8009640 con 0x7fcef80463f0 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f3de3268640 1 -- 172.21.15.160:0/2498535576 shutdown_connections 2024-05-09T07:14:04.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f5c7d653640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 0x7f5c700078c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f3de3268640 1 -- 172.21.15.160:0/2498535576 wait complete. 2024-05-09T07:14:04.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f03052ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030006edb0 0x7f03001f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f5c7d653640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5c70046d10 0x7f5c7003f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f5c767fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5c70046d10 0x7f5c7003f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f5c767fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f5c70046d10 0x7f5c7003f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44028/0 (socket says 172.21.15.160:44028) 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 0x7f5c700078c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 0x7f5c700078c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44308/0 (socket says 172.21.15.160:44308) 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcefeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80463f0 0x7fcef8183fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcefeffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80463f0 0x7fcef8183fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44302/0 (socket says 172.21.15.160:44302) 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcefeffd640 1 -- 172.21.15.160:0/2799321965 learned_addr learned my addr 172.21.15.160:0/2799321965 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fceff7fe640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef80f9450 0x7fcef8191830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7fcefe7fc640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f56a0 0x7fcef8184500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7f03052ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03000734d0 0x7f03001f7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcefe7fc640 1 -- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80463f0 msgr2=0x7fcef8183fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcefe7fc640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcef80463f0 0x7fcef8183fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcefe7fc640 1 -- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef80f9450 msgr2=0x7fcef8191830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcefe7fc640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef80f9450 0x7fcef8191830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f02feffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030006edb0 0x7f03001f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f119e7fc640 1 --2- 172.21.15.160:0/652156649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a00c3680 0x7f11a0159590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f119401a870 tx=0x7f119403b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d196359472e9868 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcefe7fc640 1 -- 172.21.15.160:0/2799321965 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8184a40 con 0x7fcef80f56a0 2024-05-09T07:14:04.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f02feffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030006edb0 0x7f03001f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44026/0 (socket says 172.21.15.160:44026) 2024-05-09T07:14:04.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f03052ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f030015cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fceff7fe640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcef80f9450 0x7fcef8191830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:04.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f03052ba640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0300159ef0 con 0x7f030006edb0 2024-05-09T07:14:04.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f03052ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0300159d70 con 0x7f03000734d0 2024-05-09T07:14:04.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f03052ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f030015a070 con 0x7f0300156e30 2024-05-09T07:14:04.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f02feffd640 1 -- 172.21.15.160:0/634594450 learned_addr learned my addr 172.21.15.160:0/634594450 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcefe7fc640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f56a0 0x7fcef8184500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf0006a880 tx=0x7fcf00069010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f5c777fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f5c7003f580 0x7f5c700e2060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36338/0 (socket says 172.21.15.160:36338) 2024-05-09T07:14:04.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f119dffb640 1 -- 172.21.15.160:0/652156649 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194048070 con 0x7f11a00c3680 2024-05-09T07:14:04.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f119dffb640 1 -- 172.21.15.160:0/652156649 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f119403bda0 con 0x7f11a00c3680 2024-05-09T07:14:04.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f119dffb640 1 -- 172.21.15.160:0/652156649 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194043660 con 0x7f11a00c3680 2024-05-09T07:14:04.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f5c7d653640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5c700deb60 con 0x7f5c7003f580 2024-05-09T07:14:04.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f5c7d653640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f5c700dc2d0 con 0x7f5c70046d10 2024-05-09T07:14:04.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f5c7d653640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5c700dee40 con 0x7f5c700dbef0 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.021+0000 7ff09670e640 1 -- 172.21.15.160:0/3046866396 shutdown_connections 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.022+0000 7ff09670e640 1 -- 172.21.15.160:0/3046866396 wait complete. 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.023+0000 7ff09670e640 1 Processor -- start 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.023+0000 7ff09670e640 1 -- start start 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7ff09670e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c006a10 0x7ff0880d5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7ff09670e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0880d6d10 0x7ff088047760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7ff09670e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff088170f60 0x7ff08816e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7ff09670e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ff0880d7f00 con 0x7ff088170f60 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7ff09670e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ff08804b0a0 con 0x7ff07c006a10 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.024+0000 7ff09670e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ff0880d8080 con 0x7ff0880d6d10 2024-05-09T07:14:04.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff094f0b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0880d6d10 0x7ff088047760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff094f0b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0880d6d10 0x7ff088047760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44320/0 (socket says 172.21.15.160:44320) 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff094f0b640 1 -- 172.21.15.160:0/1219735782 learned_addr learned my addr 172.21.15.160:0/1219735782 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff09570c640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c006a10 0x7ff0880d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff095f0d640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff088170f60 0x7ff08816e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff094f0b640 1 -- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c006a10 msgr2=0x7ff0880d5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff094f0b640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ff07c006a10 0x7ff0880d5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7ff094f0b640 1 -- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff088170f60 msgr2=0x7ff08816e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f5c767fc640 1 -- 172.21.15.160:0/1036993325 learned_addr learned my addr 172.21.15.160:0/1036993325 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7f02fffff640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f030015cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f3de3268640 1 Processor -- start 2024-05-09T07:14:04.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7ff094f0b640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff088170f60 0x7ff08816e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7ff094f0b640 1 -- 172.21.15.160:0/1219735782 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08816f6d0 con 0x7ff0880d6d10 2024-05-09T07:14:04.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7ff095f0d640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff088170f60 0x7ff08816e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:04.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f5c76ffd640 1 -- 172.21.15.160:0/1036993325 >> v2:172.21.15.160:3300/0 conn(0x7f5c70046d10 msgr2=0x7f5c7003f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f5c76ffd640 1 --2- 172.21.15.160:0/1036993325 >> v2:172.21.15.160:3300/0 conn(0x7f5c70046d10 0x7f5c7003f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f5c76ffd640 1 -- 172.21.15.160:0/1036993325 >> v1:172.21.15.160:6789/0 conn(0x7f5c7003f580 legacy=0x7f5c700e2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f5c76ffd640 1 -- 172.21.15.160:0/1036993325 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c700df120 con 0x7f5c700dbef0 2024-05-09T07:14:04.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f02feffd640 1 -- 172.21.15.160:0/634594450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 msgr2=0x7f030015cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.025+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0009d8c0 con 0x7fcef80f56a0 2024-05-09T07:14:04.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf000872e0 con 0x7fcef80f56a0 2024-05-09T07:14:04.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7fcf058d4640 1 -- 172.21.15.160:0/2799321965 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef80444d0 con 0x7fcef80f56a0 2024-05-09T07:14:04.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7fcf058d4640 1 -- 172.21.15.160:0/2799321965 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef8044a90 con 0x7fcef80f56a0 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f02feffd640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0300156e30 0x7f030015cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f02feffd640 1 -- 172.21.15.160:0/634594450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03000734d0 msgr2=0x7f03001f7500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f02feffd640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f03000734d0 0x7f03001f7500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f02feffd640 1 -- 172.21.15.160:0/634594450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03001f7f80 con 0x7f030006edb0 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f02feffd640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030006edb0 0x7f03001f4ba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02f800efc0 tx=0x7f02f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f800cda0 con 0x7f030006edb0 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f119ffff640 1 -- 172.21.15.160:0/652156649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a00c3680 msgr2=0x7f11a0159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f119ffff640 1 --2- 172.21.15.160:0/652156649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a00c3680 0x7f11a0159590 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f119401a870 tx=0x7f119403b110 comp rx=0 tx=0).stop 2024-05-09T07:14:04.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.026+0000 7f3de3268640 1 -- start start 2024-05-09T07:14:04.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f119ffff640 1 -- 172.21.15.160:0/652156649 shutdown_connections 2024-05-09T07:14:04.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f119ffff640 1 --2- 172.21.15.160:0/652156649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a00c3680 0x7f11a0159590 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f119ffff640 1 -- 172.21.15.160:0/652156649 >> 172.21.15.160:0/652156649 conn(0x7f11a00c3e00 msgr2=0x7f11a00c41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f8019070 con 0x7f030006edb0 2024-05-09T07:14:04.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f119ffff640 1 -- 172.21.15.160:0/652156649 shutdown_connections 2024-05-09T07:14:04.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f119ffff640 1 -- 172.21.15.160:0/652156649 wait complete. 2024-05-09T07:14:04.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de3268640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3ddc14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f8e9e4fc640 1 Processor -- start 2024-05-09T07:14:04.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f03052ba640 1 -- 172.21.15.160:0/634594450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03001f8260 con 0x7f030006edb0 2024-05-09T07:14:04.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f03052ba640 1 -- 172.21.15.160:0/634594450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03001586c0 con 0x7f030006edb0 2024-05-09T07:14:04.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de3268640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ddc15a0d0 0x7f3ddc1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f5c76ffd640 1 --2- 172.21.15.160:0/1036993325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 0x7f5c700078c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c680097a0 tx=0x7f5c6802a0d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4605788d0a6dcc1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f8e9e4fc640 1 -- start start 2024-05-09T07:14:04.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7ff094f0b640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0880d6d10 0x7ff088047760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff090050dd0 tx=0x7ff090069010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff09004fb60 con 0x7ff0880d6d10 2024-05-09T07:14:04.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff09006e040 con 0x7ff0880d6d10 2024-05-09T07:14:04.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de3268640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ddc1f3610 0x7f3ddc0705b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de3268640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc15d900 con 0x7f3ddc15a0d0 2024-05-09T07:14:04.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de3268640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc15d780 con 0x7f3ddc1f3610 2024-05-09T07:14:04.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de3268640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3ddc15da80 con 0x7f3dc8004800 2024-05-09T07:14:04.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f5c75ffb640 1 -- 172.21.15.160:0/1036993325 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68037070 con 0x7f5c700dbef0 2024-05-09T07:14:04.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f5c75ffb640 1 -- 172.21.15.160:0/1036993325 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c6802ade0 con 0x7f5c700dbef0 2024-05-09T07:14:04.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f5c75ffb640 1 -- 172.21.15.160:0/1036993325 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68032660 con 0x7f5c700dbef0 2024-05-09T07:14:04.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de1a65640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ddc15a0d0 0x7f3ddc1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de1a65640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ddc15a0d0 0x7f3ddc1f30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44044/0 (socket says 172.21.15.160:44044) 2024-05-09T07:14:04.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7f3de1a65640 1 -- 172.21.15.160:0/4172134787 learned_addr learned my addr 172.21.15.160:0/4172134787 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf0009d8c0 con 0x7fcef80f56a0 2024-05-09T07:14:04.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.027+0000 7fcf058d4640 1 -- 172.21.15.160:0/2799321965 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcef80053c0 con 0x7fcef80f56a0 2024-05-09T07:14:04.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.028+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0900b85e0 con 0x7ff0880d6d10 2024-05-09T07:14:04.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.028+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0880db180 con 0x7ff0880d6d10 2024-05-09T07:14:04.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.028+0000 7f5c7d653640 1 -- 172.21.15.160:0/1036993325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 msgr2=0x7f5c700078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.028+0000 7f5c7d653640 1 --2- 172.21.15.160:0/1036993325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 0x7f5c700078c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c680097a0 tx=0x7f5c6802a0d0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.028+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0880db420 con 0x7ff0880d6d10 2024-05-09T07:14:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9e4fc640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f8e98152a30 0x7f8e98155f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9e4fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8e980c8bc0 0x7f8e980c5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f119ffff640 1 Processor -- start 2024-05-09T07:14:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9e4fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9e4fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8e980c7870 con 0x7f8e980c8bc0 2024-05-09T07:14:04.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9e4fc640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f8e980c7b50 con 0x7f8e98152a30 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9ca99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9ca99640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44344/0 (socket says 172.21.15.160:44344) 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9ca99640 1 -- 172.21.15.160:0/2437123905 learned_addr learned my addr 172.21.15.160:0/2437123905 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2437123905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8e980c7e30 con 0x7f8e980c5b80 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f80146a0 con 0x7f030006edb0 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f03001f84a0 con 0x7f030006edb0 2024-05-09T07:14:04.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f119ffff640 1 -- start start 2024-05-09T07:14:04.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf00087490 con 0x7fcef80f56a0 2024-05-09T07:14:04.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7fcee7fff640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fceec06c230 0x7fceec06e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f5c7d653640 1 -- 172.21.15.160:0/1036993325 shutdown_connections 2024-05-09T07:14:04.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f5c7d653640 1 --2- 172.21.15.160:0/1036993325 >> v2:172.21.15.160:3300/0 conn(0x7f5c70046d10 0x7f5c7003f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e97fff640 1 --2- 172.21.15.160:0/2437123905 >> v2:172.21.15.188:3300/0 conn(0x7f8e98152a30 0x7f8e98155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f5c7d653640 1 --2- 172.21.15.160:0/1036993325 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c700dbef0 0x7f5c700078c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f5c7d653640 1 -- 172.21.15.160:0/1036993325 >> 172.21.15.160:0/1036993325 conn(0x7f5c70048af0 msgr2=0x7f5c70048f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e9ca99640 1 -- 172.21.15.160:0/2437123905 >> v2:172.21.15.188:3300/0 conn(0x7f8e98152a30 msgr2=0x7f8e98155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e9ca99640 1 --2- 172.21.15.160:0/2437123905 >> v2:172.21.15.188:3300/0 conn(0x7f8e98152a30 0x7f8e98155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e9ca99640 1 -- 172.21.15.160:0/2437123905 >> v2:172.21.15.160:3300/0 conn(0x7f8e980c8bc0 msgr2=0x7f8e980c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e9ca99640 1 --2- 172.21.15.160:0/2437123905 >> v2:172.21.15.160:3300/0 conn(0x7f8e980c8bc0 0x7f8e980c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e9ca99640 1 -- 172.21.15.160:0/2437123905 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e980c8110 con 0x7f8e980c5b80 2024-05-09T07:14:04.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f8e97fff640 1 --2- 172.21.15.160:0/2437123905 >> v2:172.21.15.188:3300/0 conn(0x7f8e98152a30 0x7f8e98155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:04.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ddc1f3610 0x7f3ddc0705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2266640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3ddc14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 -- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 msgr2=0x7f3ddc14ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff0880d4fc0 con 0x7ff0880d6d10 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3dc8004800 0x7f3ddc14ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 -- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ddc15a0d0 msgr2=0x7f3ddc1f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ddc15a0d0 0x7f3ddc1f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 -- 172.21.15.160:0/4172134787 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc1f2910 con 0x7f3ddc1f3610 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.029+0000 7f3de2a67640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ddc1f3610 0x7f3ddc0705b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc04fca0 tx=0x7f3dd4004840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4019070 con 0x7f3ddc1f3610 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3dd4007cd0 con 0x7f3ddc1f3610 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4014640 con 0x7f3ddc1f3610 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f3de3268640 1 -- 172.21.15.160:0/4172134787 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc1575f0 con 0x7f3ddc1f3610 2024-05-09T07:14:04.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f3de3268640 1 -- 172.21.15.160:0/4172134787 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc157bb0 con 0x7f3ddc1f3610 2024-05-09T07:14:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f5c7d653640 1 -- 172.21.15.160:0/1036993325 shutdown_connections 2024-05-09T07:14:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119ffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a0070f20 0x7f11a006dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119ffff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a00c3680 0x7f11a0158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a0156b30 0x7f11a00c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf0009d6c0 con 0x7fcef80f56a0 2024-05-09T07:14:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119ffff640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f11a015a1e0 con 0x7f11a0156b30 2024-05-09T07:14:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119ffff640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f11a0156f10 con 0x7f11a00c3680 2024-05-09T07:14:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119ffff640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f11a006f6a0 con 0x7f11a0070f20 2024-05-09T07:14:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a0156b30 0x7f11a00c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a0156b30 0x7f11a00c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44070/0 (socket says 172.21.15.160:44070) 2024-05-09T07:14:04.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7f119f7fe640 1 -- 172.21.15.160:0/1336727465 learned_addr learned my addr 172.21.15.160:0/1336727465 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f119f7fe640 1 -- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a0070f20 msgr2=0x7f11a006dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f119f7fe640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f11a0070f20 0x7f11a006dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f119f7fe640 1 -- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a00c3680 msgr2=0x7f11a0158450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T07:14:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f119f7fe640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f11a00c3680 0x7f11a0158450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f5c7d653640 1 -- 172.21.15.160:0/1036993325 wait complete. 2024-05-09T07:14:04.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f8e9ca99640 1 --2- 172.21.15.160:0/2437123905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98159020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8e9808c980 tx=0x7f8e9002a6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e85a49df5dea37ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f119f7fe640 1 -- 172.21.15.160:0/1336727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a0159040 con 0x7f11a0156b30 2024-05-09T07:14:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f8e96ffd640 1 -- 172.21.15.160:0/2437123905 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e90036070 con 0x7f8e980c5b80 2024-05-09T07:14:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f20b0e1c640 1 Processor -- start 2024-05-09T07:14:04.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f8e96ffd640 1 -- 172.21.15.160:0/2437123905 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e90031600 con 0x7f8e980c5b80 2024-05-09T07:14:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f8e96ffd640 1 -- 172.21.15.160:0/2437123905 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e900319b0 con 0x7f8e980c5b80 2024-05-09T07:14:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7f119f7fe640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a0156b30 0x7f11a00c4d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f119800c910 tx=0x7f119800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ff0900b7040 con 0x7ff0880d6d10 2024-05-09T07:14:04.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198017070 con 0x7f11a0156b30 2024-05-09T07:14:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f20b0e1c640 1 -- start start 2024-05-09T07:14:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f119ffff640 1 -- 172.21.15.160:0/1336727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a00c5820 con 0x7f11a0156b30 2024-05-09T07:14:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f119ffff640 1 -- 172.21.15.160:0/1336727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a00c69c0 con 0x7f11a0156b30 2024-05-09T07:14:04.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f02f8012050 con 0x7f030006edb0 2024-05-09T07:14:04.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7ff085ffb640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff07406beb0 0x7ff07406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7fcefeffd640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fceec06c230 0x7fceec06e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2437123905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 msgr2=0x7f8e98159020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f8e9e4fc640 1 --2- 172.21.15.160:0/2437123905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98159020 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8e9808c980 tx=0x7f8e9002a6b0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2437123905 shutdown_connections 2024-05-09T07:14:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f8e9e4fc640 1 --2- 172.21.15.160:0/2437123905 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98159020 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7ff09570c640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff07406beb0 0x7ff07406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f8e9e4fc640 1 --2- 172.21.15.160:0/2437123905 >> v2:172.21.15.160:3300/0 conn(0x7f8e980c8bc0 0x7f8e980c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7fcee7fff640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7fceec076500 0x7fceec0789a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f8e9e4fc640 1 --2- 172.21.15.160:0/2437123905 >> v2:172.21.15.188:3300/0 conn(0x7f8e98152a30 0x7f8e98155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2437123905 >> 172.21.15.160:0/2437123905 conn(0x7f8e98072ec0 msgr2=0x7f8e980ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.030+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 --> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fceec079070 con 0x7fceec076500 2024-05-09T07:14:04.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7fcefeffd640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fceec06c230 0x7fceec06e6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcef0001980 tx=0x7fcef0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f02e77fe640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f02ec06bf30 0x7f02ec06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f5c7d653640 1 Processor -- start 2024-05-09T07:14:04.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.031+0000 7fceff7fe640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7fceec076500 0x7fceec0789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcf000e0000 con 0x7fcef80f56a0 2024-05-09T07:14:04.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7fceff7fe640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7fceec076500 0x7fceec0789a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f5c7d653640 1 -- start start 2024-05-09T07:14:04.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f02f808c500 con 0x7f030006edb0 2024-05-09T07:14:04.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== osd.7 v2:172.21.15.188:6808/4113254918 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fceec079070 con 0x7fceec076500 2024-05-09T07:14:04.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.032+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3dd401c070 con 0x7f3ddc1f3610 2024-05-09T07:14:04.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f3dd2ffd640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3dc006bf00 0x7f3dc006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3dd408ac80 con 0x7f3ddc1f3610 2024-05-09T07:14:04.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f02e77fe640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f02ec0756a0 0x7f02ec077ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff0900bf070 con 0x7ff0880d6d10 2024-05-09T07:14:04.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 --> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f02f8012e50 con 0x7f02ec0756a0 2024-05-09T07:14:04.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f3de2266640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3dc006bf00 0x7f3dc006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f3dd0ff9640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3db8001650 0x7f3db8003b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f02f808c8f0 con 0x7f030006edb0 2024-05-09T07:14:04.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 --> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3db8006bb0 con 0x7f3db8001650 2024-05-09T07:14:04.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f02fe7fc640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f02ec06bf30 0x7f02ec06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20b0e1c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 0x7f20ac0c5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20b0e1c640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20ac0c63c0 0x7f20ac156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20b0e1c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20ac06e280 con 0x7f20ac157520 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20b0e1c640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f20ac0c8450 con 0x7f20ac0c63c0 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20b0e1c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f20ac0c86a0 con 0x7f20ac0c4140 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20aa59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 0x7f20ac0c5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20aa59c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 0x7f20ac0c5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44360/0 (socket says 172.21.15.160:44360) 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7f20aa59c640 1 -- 172.21.15.160:0/216303414 learned_addr learned my addr 172.21.15.160:0/216303414 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.088 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2437123905 shutdown_connections 2024-05-09T07:14:04.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7ff085ffb640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7ff074075620 0x7ff074077a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 --> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff0900b8e50 con 0x7ff074075620 2024-05-09T07:14:04.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.033+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff090126d80 con 0x7ff0880d6d10 2024-05-09T07:14:04.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7ff095f0d640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7ff074075620 0x7ff074077a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2437123905 wait complete. 2024-05-09T07:14:04.089 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1198003dc0 con 0x7f11a0156b30 2024-05-09T07:14:04.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119800a940 con 0x7f11a0156b30 2024-05-09T07:14:04.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f20aa59c640 1 -- 172.21.15.160:0/216303414 >> v2:172.21.15.188:3300/0 conn(0x7f20ac0c63c0 msgr2=0x7f20ac156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f20a9d9b640 1 --2- 172.21.15.160:0/216303414 >> v2:172.21.15.188:3300/0 conn(0x7f20ac0c63c0 0x7f20ac156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f20aa59c640 1 --2- 172.21.15.160:0/216303414 >> v2:172.21.15.188:3300/0 conn(0x7f20ac0c63c0 0x7f20ac156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f20aa59c640 1 -- 172.21.15.160:0/216303414 >> v1:172.21.15.160:6789/0 conn(0x7f20ac157520 legacy=0x7f20ac06f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.090 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f20aa59c640 1 -- 172.21.15.160:0/216303414 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac0c8980 con 0x7f20ac0c4140 2024-05-09T07:14:04.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f5c7d653640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c7003f580 0x7f5c700dad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f5c7d653640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c70046d10 0x7f5c70041d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.091 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f5c7d653640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c700dbef0 0x7f5c70174aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c7003f580 0x7f5c700dad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c76ffd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c7003f580 0x7f5c700dad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44364/0 (socket says 172.21.15.160:44364) 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c76ffd640 1 -- 172.21.15.160:0/1348307523 learned_addr learned my addr 172.21.15.160:0/1348307523 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700de810 con 0x7f5c700dbef0 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c700dbef0 0x7f5c70174aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5c700df6e0 con 0x7f5c70046d10 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5c700e2770 con 0x7f5c7003f580 2024-05-09T07:14:04.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f20a9d9b640 1 --2- 172.21.15.160:0/216303414 >> v2:172.21.15.188:3300/0 conn(0x7f20ac0c63c0 0x7f20ac156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c767fc640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c70046d10 0x7f5c70041d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f20aa59c640 1 --2- 172.21.15.160:0/216303414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 0x7f20ac0c5400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c12e7b9120ae418f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f20a959a640 1 -- 172.21.15.160:0/216303414 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c037070 con 0x7f20ac0c4140 2024-05-09T07:14:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f20a959a640 1 -- 172.21.15.160:0/216303414 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209c032440 con 0x7f20ac0c4140 2024-05-09T07:14:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f20a959a640 1 -- 172.21.15.160:0/216303414 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c032810 con 0x7f20ac0c4140 2024-05-09T07:14:04.093 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f8e9e4fc640 1 Processor -- start 2024-05-09T07:14:04.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7ff095f0d640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7ff074075620 0x7ff074077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f119ffff640 1 -- 172.21.15.160:0/1336727465 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1180001030 con 0x7f11a0156b30 2024-05-09T07:14:04.094 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 -- 172.21.15.160:0/1348307523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c7003f580 msgr2=0x7f5c700dad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c7003f580 0x7f5c700dad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 -- 172.21.15.160:0/1348307523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c70046d10 msgr2=0x7f5c70041d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5c70046d10 0x7f5c70041d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c701745c0 con 0x7f5c700dbef0 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c76ffd640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5c7003f580 0x7f5c700dad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c777fe640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c700dbef0 0x7f5c70174aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c78069bf0 tx=0x7f5c78068d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780945a0 con 0x7f5c700dbef0 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c700e18f0 con 0x7f5c700dbef0 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c701788b0 con 0x7f5c700dbef0 2024-05-09T07:14:04.095 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7ff09570c640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff07406beb0 0x7ff07406e370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009a30 tx=0x7ff08c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f119801a070 con 0x7f11a0156b30 2024-05-09T07:14:04.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f02fffff640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f02ec0756a0 0x7f02ec077ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.096 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5c7806e040 con 0x7f5c700dbef0 2024-05-09T07:14:04.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.034+0000 7f3de1a65640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3db8001650 0x7f3db8003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.035+0000 7f3de1a65640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3db8001650 0x7f3db8003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== osd.6 v2:172.21.15.188:6824/566542668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3db8006bb0 con 0x7f3db8001650 2024-05-09T07:14:04.097 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== osd.4 v2:172.21.15.188:6800/2865253491 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff0900b8e50 con 0x7ff074075620 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.036+0000 7f11877fe640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f118c06bf30 0x7f118c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f119808a8f0 con 0x7f11a0156b30 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f119effd640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f118c06bf30 0x7f118c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f11877fe640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7f118c0756b0 0x7f118c077af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 --> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f118c0781c0 con 0x7f118c0756b0 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f20b0e1c640 1 -- 172.21.15.160:0/216303414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 msgr2=0x7f20ac0c5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/216303414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 0x7f20ac0c5400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c02a210 comp rx=0 tx=0).stop 2024-05-09T07:14:04.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f20b0e1c640 1 -- 172.21.15.160:0/216303414 shutdown_connections 2024-05-09T07:14:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/216303414 >> v2:172.21.15.188:3300/0 conn(0x7f20ac0c63c0 0x7f20ac156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/216303414 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac0c4140 0x7f20ac0c5400 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c02a210 comp rx=0 tx=0).stop 2024-05-09T07:14:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f20b0e1c640 1 -- 172.21.15.160:0/216303414 >> 172.21.15.160:0/216303414 conn(0x7f20ac066f90 msgr2=0x7f20ac067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f119e7fc640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7f118c0756b0 0x7f118c077af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.099 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f119808f2a0 con 0x7f11a0156b30 2024-05-09T07:14:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780949c0 con 0x7f5c700dbef0 2024-05-09T07:14:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5c58001030 con 0x7f5c700dbef0 2024-05-09T07:14:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f119e7fc640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7f118c0756b0 0x7f118c077af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f119effd640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f118c06bf30 0x7f118c06e3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11900014d0 tx=0x7f1190007a50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f02fe7fc640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f02ec06bf30 0x7f02ec06e3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02f4034000 tx=0x7f02f402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f02fffff640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f02ec0756a0 0x7f02ec077ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.037+0000 7f8e9e4fc640 1 -- start start 2024-05-09T07:14:04.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f3de2266640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3dc006bf00 0x7f3dc006e3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0699a0 tx=0x7f3dd8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f20b0e1c640 1 -- 172.21.15.160:0/216303414 shutdown_connections 2024-05-09T07:14:04.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5c78094c20 con 0x7f5c700dbef0 2024-05-09T07:14:04.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.038+0000 7f20b0e1c640 1 -- 172.21.15.160:0/216303414 wait complete. 2024-05-09T07:14:04.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== osd.0 v2:172.21.15.160:6800/3071505742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f02f8012e50 con 0x7f02ec0756a0 2024-05-09T07:14:04.102 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== osd.5 v2:172.21.15.188:6816/173821344 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f118c0781c0 con 0x7f118c0756b0 2024-05-09T07:14:04.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f5c5f7fe640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5c6406be30 0x7f5c6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f8e9e4fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f8e9e4fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980c8bc0 0x7f8e980c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f8e9e4fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e98152a30 0x7f8e98147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f8e9e4fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8e980c90d0 con 0x7f8e98152a30 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f8e9e4fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8e98152e10 con 0x7f8e980c8bc0 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.039+0000 7f8e9e4fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8e980c9250 con 0x7f8e980c5b80 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980c8bc0 0x7f8e980c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5c78106c60 con 0x7f5c700dbef0 2024-05-09T07:14:04.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980c8bc0 0x7f8e980c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53436/0 (socket says 172.21.15.160:53436) 2024-05-09T07:14:04.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 -- 172.21.15.160:0/2576822732 learned_addr learned my addr 172.21.15.160:0/2576822732 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f5c76ffd640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5c6406be30 0x7f5c6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f5c5f7fe640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7f5c640755b0 0x7f5c640779f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5c7806e040 con 0x7f5c640755b0 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5c781092c0 con 0x7f5c700dbef0 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 -- 172.21.15.160:0/2576822732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 msgr2=0x7f8e98154ac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8e980c5b80 0x7f8e98154ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 -- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e98152a30 msgr2=0x7f8e98147e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8e98152a30 0x7f8e98147e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f8e977fe640 1 -- 172.21.15.160:0/2576822732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e98155000 con 0x7f8e980c8bc0 2024-05-09T07:14:04.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f20b0e1c640 1 Processor -- start 2024-05-09T07:14:04.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.041+0000 7f8e977fe640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980c8bc0 0x7f8e980c9e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b4a0 tx=0x7f8e8800b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.040+0000 7f20b0e1c640 1 -- start start 2024-05-09T07:14:04.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.041+0000 7f5c76ffd640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5c6406be30 0x7f5c6406e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c68000c00 tx=0x7f5c68008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.041+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88014070 con 0x7f8e980c8bc0 2024-05-09T07:14:04.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.041+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e88005a10 con 0x7f8e980c8bc0 2024-05-09T07:14:04.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.041+0000 7f5c767fc640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7f5c640755b0 0x7f5c640779f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.108 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e88005de0 con 0x7f8e980c8bc0 2024-05-09T07:14:04.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2576822732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e981f2cd0 con 0x7f8e980c8bc0 2024-05-09T07:14:04.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2576822732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e981f3290 con 0x7f8e980c8bc0 2024-05-09T07:14:04.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f5c767fc640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7f5c640755b0 0x7f5c640779f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.041+0000 7f20b0e1c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ac0c63c0 0x7f20ac149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ac0c63c0 0x7f20ac149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ac0c63c0 0x7f20ac149de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44078/0 (socket says 172.21.15.160:44078) 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 -- 172.21.15.160:0/57783979 learned_addr learned my addr 172.21.15.160:0/57783979 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac157520 0x7f20ac0a7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac147920 0x7f20ac0a7540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac0c9220 con 0x7f20ac0c63c0 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac0c8f40 con 0x7f20ac147920 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f20ac0c4950 con 0x7f20ac157520 2024-05-09T07:14:04.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aad9d640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac147920 0x7f20ac0a7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20a9d9b640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac157520 0x7f20ac0a7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 -- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac157520 msgr2=0x7f20ac0a7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20ac157520 0x7f20ac0a7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 -- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac147920 msgr2=0x7f20ac0a7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.042+0000 7f20aa59c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f20ac147920 0x7f20ac0a7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.043+0000 7f20aa59c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac147ef0 con 0x7f20ac0c63c0 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.043+0000 7f20aa59c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ac0c63c0 0x7f20ac149de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c041020 con 0x7f20ac0c63c0 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f209c030990 con 0x7f20ac0c63c0 2024-05-09T07:14:04.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2576822732 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e980c6780 con 0x7f8e980c8bc0 2024-05-09T07:14:04.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8e88010610 con 0x7f8e980c8bc0 2024-05-09T07:14:04.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== osd.2 v2:172.21.15.160:6808/2850827028 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5c7806e040 con 0x7f5c640755b0 2024-05-09T07:14:04.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c037040 con 0x7f20ac0c63c0 2024-05-09T07:14:04.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.044+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac0a8f60 con 0x7f20ac0c63c0 2024-05-09T07:14:04.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.045+0000 7f8e94ff9640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8e8406be80 0x7f8e8406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.045+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac14a320 con 0x7f20ac0c63c0 2024-05-09T07:14:04.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.045+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8e88056ac0 con 0x7f8e980c8bc0 2024-05-09T07:14:04.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.045+0000 7f8e94ff9640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f8e84076250 0x7f8e840786f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.045+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 --> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e88005a10 con 0x7f8e84076250 2024-05-09T07:14:04.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.045+0000 7f8e97fff640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8e8406be80 0x7f8e8406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.046+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20ac0a8730 con 0x7f20ac0c63c0 2024-05-09T07:14:04.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.046+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8e88055450 con 0x7f8e980c8bc0 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.046+0000 7f8e97fff640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8e8406be80 0x7f8e8406e340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00a810 tx=0x7f8e8c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.046+0000 7f8e9ca99640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f8e84076250 0x7f8e840786f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.047+0000 7f8e9ca99640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f8e84076250 0x7f8e840786f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.047+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f209c044020 con 0x7f20ac0c63c0 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.048+0000 7f209affd640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f208c06beb0 0x7f208c06e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.048+0000 7f20a9d9b640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f208c06beb0 0x7f208c06e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.049+0000 7f20a9d9b640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f208c06beb0 0x7f208c06e370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f20ac169960 tx=0x7f20a0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.049+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f209c0abf00 con 0x7f20ac0c63c0 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.049+0000 7f209affd640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f208c075630 0x7f208c077a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.049+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f209c031ec0 con 0x7f208c075630 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.049+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f209c0ac3e0 con 0x7f20ac0c63c0 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.049+0000 7f20aad9d640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f208c075630 0x7f208c077a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.050+0000 7f20aad9d640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f208c075630 0x7f208c077a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.051+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== osd.1 v2:172.21.15.160:6816/4171986763 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f209c031ec0 con 0x7f208c075630 2024-05-09T07:14:04.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.048+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== osd.3 v2:172.21.15.160:6824/3093825945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e88005a10 con 0x7f8e84076250 2024-05-09T07:14:04.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.064+0000 7fcf058d4640 1 -- 172.21.15.160:0/2799321965 --> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcef80053c0 con 0x7fceec076500 2024-05-09T07:14:04.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.065+0000 7fcee7fff640 1 -- 172.21.15.160:0/2799321965 <== osd.7 v2:172.21.15.188:6808/4113254918 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcef80053c0 con 0x7fceec076500 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7fceec076500 msgr2=0x7fceec0789a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7fceec076500 0x7fceec0789a0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fceec06c230 msgr2=0x7fceec06e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fceec06c230 0x7fceec06e6f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcef0001980 tx=0x7fcef0009040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f56a0 msgr2=0x7fcef8184500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 --2- 172.21.15.160:0/2799321965 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcef80f56a0 0x7fcef8184500 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fcf0006a880 tx=0x7fcf00069010 comp rx=0 tx=0).stop 2024-05-09T07:14:04.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fceff7fe640 1 -- 172.21.15.160:0/2799321965 reap_dead start 2024-05-09T07:14:04.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 shutdown_connections 2024-05-09T07:14:04.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.066+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 >> 172.21.15.160:0/2799321965 conn(0x7fcef8005ec0 msgr2=0x7fcef80049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.067+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 shutdown_connections 2024-05-09T07:14:04.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.067+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 --> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff088005390 con 0x7ff074075620 2024-05-09T07:14:04.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.067+0000 7fcee5ffb640 1 -- 172.21.15.160:0/2799321965 wait complete. 2024-05-09T07:14:04.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.068+0000 7ff085ffb640 1 -- 172.21.15.160:0/1219735782 <== osd.4 v2:172.21.15.188:6800/2865253491 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff088005390 con 0x7ff074075620 2024-05-09T07:14:04.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.068+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7ff074075620 msgr2=0x7ff074077a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.068+0000 7ff09670e640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7ff074075620 0x7ff074077a60 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.068+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 --> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3db8005c40 con 0x7f3db8001650 2024-05-09T07:14:04.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd2ffd640 1 -- 172.21.15.160:0/4172134787 <== osd.6 v2:172.21.15.188:6824/566542668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3db8005c40 con 0x7f3db8001650 2024-05-09T07:14:04.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3db8001650 msgr2=0x7f3db8003b10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd0ff9640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3db8001650 0x7f3db8003b10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3dc006bf00 msgr2=0x7f3dc006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd0ff9640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3dc006bf00 0x7f3dc006e3c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0699a0 tx=0x7f3dd8009040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ddc1f3610 msgr2=0x7f3ddc0705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.069+0000 7f3dd0ff9640 1 --2- 172.21.15.160:0/4172134787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ddc1f3610 0x7f3ddc0705b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc04fca0 tx=0x7f3dd4004840 comp rx=0 tx=0).stop 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff07406beb0 msgr2=0x7ff07406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ff07406beb0 0x7ff07406e370 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff08c009a30 tx=0x7ff08c008040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0880d6d10 msgr2=0x7ff088047760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 --2- 172.21.15.160:0/1219735782 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ff0880d6d10 0x7ff088047760 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff090050dd0 tx=0x7ff090069010 comp rx=0 tx=0).stop 2024-05-09T07:14:04.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff095f0d640 1 -- 172.21.15.160:0/1219735782 reap_dead start 2024-05-09T07:14:04.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f3de2a67640 1 -- 172.21.15.160:0/4172134787 reap_dead start 2024-05-09T07:14:04.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 shutdown_connections 2024-05-09T07:14:04.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 >> 172.21.15.160:0/4172134787 conn(0x7f3ddc0a85a0 msgr2=0x7f3ddc0c15c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 shutdown_connections 2024-05-09T07:14:04.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 >> 172.21.15.160:0/1219735782 conn(0x7ff088005e90 msgr2=0x7ff0880cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 shutdown_connections 2024-05-09T07:14:04.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 shutdown_connections 2024-05-09T07:14:04.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f3dd0ff9640 1 -- 172.21.15.160:0/4172134787 wait complete. 2024-05-09T07:14:04.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7ff09670e640 1 -- 172.21.15.160:0/1219735782 wait complete. 2024-05-09T07:14:04.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f119ffff640 1 -- 172.21.15.160:0/1336727465 --> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1180002dd0 con 0x7f118c0756b0 2024-05-09T07:14:04.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.070+0000 7f11877fe640 1 -- 172.21.15.160:0/1336727465 <== osd.5 v2:172.21.15.188:6816/173821344 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1180002dd0 con 0x7f118c0756b0 2024-05-09T07:14:04.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.071+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7f118c0756b0 msgr2=0x7f118c077af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.071+0000 7f11857fa640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7f118c0756b0 0x7f118c077af0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f118c06bf30 msgr2=0x7f118c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f11857fa640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f118c06bf30 0x7f118c06e3f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f11900014d0 tx=0x7f1190007a50 comp rx=0 tx=0).stop 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a0156b30 msgr2=0x7f11a00c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f11857fa640 1 --2- 172.21.15.160:0/1336727465 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f11a0156b30 0x7f11a00c4d10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f119800c910 tx=0x7f119800cdd0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f119f7fe640 1 -- 172.21.15.160:0/1336727465 reap_dead start 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 --> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f03000756c0 con 0x7f02ec0756a0 2024-05-09T07:14:04.127 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.072+0000 7f02e77fe640 1 -- 172.21.15.160:0/634594450 <== osd.0 v2:172.21.15.160:6800/3071505742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f03000756c0 con 0x7f02ec0756a0 2024-05-09T07:14:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 shutdown_connections 2024-05-09T07:14:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 >> 172.21.15.160:0/1336727465 conn(0x7f11a00c3e00 msgr2=0x7f11a0155090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 shutdown_connections 2024-05-09T07:14:04.128 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f11857fa640 1 -- 172.21.15.160:0/1336727465 wait complete. 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f02ec0756a0 msgr2=0x7f02ec077ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02e57fa640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f02ec0756a0 0x7f02ec077ae0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f02ec06bf30 msgr2=0x7f02ec06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02e57fa640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f02ec06bf30 0x7f02ec06e3f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f02f4034000 tx=0x7f02f402e000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030006edb0 msgr2=0x7f03001f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02e57fa640 1 --2- 172.21.15.160:0/634594450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f030006edb0 0x7f03001f4ba0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f02f800efc0 tx=0x7f02f800c490 comp rx=0 tx=0).stop 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.073+0000 7f02fffff640 1 -- 172.21.15.160:0/634594450 reap_dead start 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.074+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 shutdown_connections 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.074+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 >> 172.21.15.160:0/634594450 conn(0x7f030006fb10 msgr2=0x7f030006fef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.074+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 shutdown_connections 2024-05-09T07:14:04.129 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.074+0000 7f02e57fa640 1 -- 172.21.15.160:0/634594450 wait complete. 2024-05-09T07:14:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.080+0000 7f5c7d653640 1 -- 172.21.15.160:0/1348307523 --> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5c58002dd0 con 0x7f5c640755b0 2024-05-09T07:14:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.080+0000 7f8e9e4fc640 1 -- 172.21.15.160:0/2576822732 --> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e9814f820 con 0x7f8e84076250 2024-05-09T07:14:04.130 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.081+0000 7f8e94ff9640 1 -- 172.21.15.160:0/2576822732 <== osd.3 v2:172.21.15.160:6824/3093825945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f8e9814f820 con 0x7f8e84076250 2024-05-09T07:14:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.081+0000 7f5c5f7fe640 1 -- 172.21.15.160:0/1348307523 <== osd.2 v2:172.21.15.160:6808/2850827028 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5c58002dd0 con 0x7f5c640755b0 2024-05-09T07:14:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7f5c640755b0 msgr2=0x7f5c640779f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.131 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f5c5d7fa640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7f5c640755b0 0x7f5c640779f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f8e84076250 msgr2=0x7f8e840786f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f8e7e7fc640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f8e84076250 0x7f8e840786f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8e8406be80 msgr2=0x7f8e8406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f8e7e7fc640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8e8406be80 0x7f8e8406e340 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8e8c00a810 tx=0x7f8e8c008040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980c8bc0 msgr2=0x7f8e980c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.082+0000 7f8e7e7fc640 1 --2- 172.21.15.160:0/2576822732 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8e980c8bc0 0x7f8e980c9e60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8e8800b4a0 tx=0x7f8e8800b960 comp rx=0 tx=0).stop 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f8e9ca99640 1 -- 172.21.15.160:0/2576822732 reap_dead start 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 shutdown_connections 2024-05-09T07:14:04.132 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 >> 172.21.15.160:0/2576822732 conn(0x7f8e98072ec0 msgr2=0x7f8e9814b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5c6406be30 msgr2=0x7f5c6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c5d7fa640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5c6406be30 0x7f5c6406e2f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5c68000c00 tx=0x7f5c68008040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c700dbef0 msgr2=0x7f5c70174aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c5d7fa640 1 --2- 172.21.15.160:0/1348307523 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c700dbef0 0x7f5c70174aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5c78069bf0 tx=0x7f5c78068d70 comp rx=0 tx=0).stop 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c777fe640 1 -- 172.21.15.160:0/1348307523 reap_dead start 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 shutdown_connections 2024-05-09T07:14:04.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f8e7e7fc640 1 -- 172.21.15.160:0/2576822732 wait complete. 2024-05-09T07:14:04.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 shutdown_connections 2024-05-09T07:14:04.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.083+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 >> 172.21.15.160:0/1348307523 conn(0x7f5c70048af0 msgr2=0x7f5c70007b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.084+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 --> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20ac0756c0 con 0x7f208c075630 2024-05-09T07:14:04.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.084+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 shutdown_connections 2024-05-09T07:14:04.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.084+0000 7f5c5d7fa640 1 -- 172.21.15.160:0/1348307523 wait complete. 2024-05-09T07:14:04.135 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.084+0000 7f209affd640 1 -- 172.21.15.160:0/57783979 <== osd.1 v2:172.21.15.160:6816/4171986763 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f20ac0756c0 con 0x7f208c075630 2024-05-09T07:14:04.136 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:04.136 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T07:14:04.136 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.089+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f208c075630 msgr2=0x7f208c077a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.089+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f208c075630 0x7f208c077a70 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.090+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f208c06beb0 msgr2=0x7f208c06e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.090+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f208c06beb0 0x7f208c06e370 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f20ac169960 tx=0x7f20a0008040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.090+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ac0c63c0 msgr2=0x7f20ac149de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.090+0000 7f20b0e1c640 1 --2- 172.21.15.160:0/57783979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20ac0c63c0 0x7f20ac149de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f209c009870 tx=0x7f209c0046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.091+0000 7f20aad9d640 1 -- 172.21.15.160:0/57783979 reap_dead start 2024-05-09T07:14:04.137 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:04.138 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:04.138 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T07:14:04.139 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T07:14:04.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.092+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 shutdown_connections 2024-05-09T07:14:04.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.092+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 >> 172.21.15.160:0/57783979 conn(0x7f20ac066f90 msgr2=0x7f20ac074170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.092+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 shutdown_connections 2024-05-09T07:14:04.140 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.092+0000 7f20b0e1c640 1 -- 172.21.15.160:0/57783979 wait complete. 2024-05-09T07:14:04.140 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:04.141 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T07:14:04.141 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:04.142 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T07:14:04.142 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:04.143 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T07:14:04.143 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:04.144 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T07:14:04.144 INFO:teuthology.orchestra.run.smithi160.stdout:55834574859 2024-05-09T07:14:04.145 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T07:14:04.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.351+0000 7f757c476640 1 Processor -- start 2024-05-09T07:14:04.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.351+0000 7f757c476640 1 -- start start 2024-05-09T07:14:04.358 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.352+0000 7f757c476640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f757406c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.352+0000 7f757c476640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f75740c38f0 0x7f75741654d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.352+0000 7f757c476640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7574161f00 con 0x7f75740c60e0 2024-05-09T07:14:04.359 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.352+0000 7f757c476640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f75740a6440 con 0x7f75740c38f0 2024-05-09T07:14:04.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.352+0000 7f757c476640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7574162ab0 con 0x7f75740c3510 2024-05-09T07:14:04.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.353+0000 7f757a212640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f757406c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.353+0000 7f757a212640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f757406c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44378/0 (socket says 172.21.15.160:44378) 2024-05-09T07:14:04.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.353+0000 7f757a212640 1 -- 172.21.15.160:0/3778409498 learned_addr learned my addr 172.21.15.160:0/3778409498 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.353+0000 7f757aa13640 1 --2- 172.21.15.160:0/3778409498 >> v2:172.21.15.188:3300/0 conn(0x7f75740c38f0 0x7f75741654d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.356+0000 7f7579210640 1 -- 172.21.15.160:0/3778409498 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2463383630 0 0) 0x7f7564001110 con 0x7f75740c60e0 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.356+0000 7f7579210640 1 -- 172.21.15.160:0/3778409498 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7560007930 con 0x7f75740c60e0 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.357+0000 7f757a212640 1 -- 172.21.15.160:0/3778409498 >> v2:172.21.15.188:3300/0 conn(0x7f75740c38f0 msgr2=0x7f75741654d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.357+0000 7f757a212640 1 --2- 172.21.15.160:0/3778409498 >> v2:172.21.15.188:3300/0 conn(0x7f75740c38f0 0x7f75741654d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.357+0000 7f757a212640 1 -- 172.21.15.160:0/3778409498 >> v1:172.21.15.188:6789/0 conn(0x7f75740c60e0 legacy=0x7f757406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.357+0000 7f757a212640 1 -- 172.21.15.160:0/3778409498 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574162180 con 0x7f75740c3510 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.358+0000 7f757a212640 1 --2- 172.21.15.160:0/3778409498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f757406c3b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7570013d10 tx=0x7f7570032b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=976605f0788a2fdc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.359+0000 7f00ebfff640 1 Processor -- start 2024-05-09T07:14:04.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.359+0000 7f00ebfff640 1 -- start start 2024-05-09T07:14:04.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.359+0000 7f7579210640 1 -- 172.21.15.160:0/3778409498 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75700348b0 con 0x7f75740c3510 2024-05-09T07:14:04.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.359+0000 7f7579210640 1 -- 172.21.15.160:0/3778409498 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f757000ae50 con 0x7f75740c3510 2024-05-09T07:14:04.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.359+0000 7f7579210640 1 -- 172.21.15.160:0/3778409498 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7570034cb0 con 0x7f75740c3510 2024-05-09T07:14:04.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.360+0000 7f757c476640 1 -- 172.21.15.160:0/3778409498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 msgr2=0x7f757406c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.360+0000 7f757c476640 1 --2- 172.21.15.160:0/3778409498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f757406c3b0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7570013d10 tx=0x7f7570032b50 comp rx=0 tx=0).stop 2024-05-09T07:14:04.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f00ebfff640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f00ec0b0350 0x7f00ec0c8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 0x7f00ec0c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f00eb7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f00ec0c59f0 0x7f00ec1588e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36354/0 (socket says 172.21.15.160:36354) 2024-05-09T07:14:04.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f00e3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 0x7f00ec0c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f00e3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 0x7f00ec0c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44386/0 (socket says 172.21.15.160:44386) 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f00eb7fe640 1 -- 172.21.15.160:0/2811674163 learned_addr learned my addr 172.21.15.160:0/2811674163 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f757c476640 1 -- 172.21.15.160:0/3778409498 shutdown_connections 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f757c476640 1 --2- 172.21.15.160:0/3778409498 >> v2:172.21.15.188:3300/0 conn(0x7f75740c38f0 0x7f75741654d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f757c476640 1 --2- 172.21.15.160:0/3778409498 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f757406c3b0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7f757c476640 1 -- 172.21.15.160:0/3778409498 >> 172.21.15.160:0/3778409498 conn(0x7f75740c68d0 msgr2=0x7f75740c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.360+0000 7feab485c640 1 Processor -- start 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.360+0000 7feab485c640 1 -- start start 2024-05-09T07:14:04.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7feab485c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0156c90 0x7feab015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7feab485c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab0157070 con 0x7feab00c2670 2024-05-09T07:14:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7feab485c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feab01595f0 con 0x7feab0073250 2024-05-09T07:14:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.361+0000 7feab485c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feab0159940 con 0x7feab0156c90 2024-05-09T07:14:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00ec06d460 con 0x7f00ec0c59f0 2024-05-09T07:14:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7feaaf59e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7feab00c2670 0x7feab00c6e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36370/0 (socket says 172.21.15.160:36370) 2024-05-09T07:14:04.365 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7feaaf59e640 1 -- 172.21.15.160:0/3303009754 learned_addr learned my addr 172.21.15.160:0/3303009754 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f00ec152d30 con 0x7f00ec0b0350 2024-05-09T07:14:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec155430 con 0x7f00ec0c8f70 2024-05-09T07:14:04.366 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7feaaed9d640 1 --2- 172.21.15.160:0/3303009754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0156c90 0x7feab015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00e3fff640 1 -- 172.21.15.160:0/2811674163 >> v2:172.21.15.188:3300/0 conn(0x7f00ec0b0350 msgr2=0x7f00ec0c8a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:14:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00e3fff640 1 --2- 172.21.15.160:0/2811674163 >> v2:172.21.15.188:3300/0 conn(0x7f00ec0b0350 0x7f00ec0c8a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00e3fff640 1 -- 172.21.15.160:0/2811674163 >> v1:172.21.15.160:6789/0 conn(0x7f00ec0c59f0 legacy=0x7f00ec1588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00ea7fc640 1 -- 172.21.15.160:0/2811674163 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3577087522 0 0) 0x7f00ec06d460 con 0x7f00ec0c59f0 2024-05-09T07:14:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.362+0000 7f00e3fff640 1 -- 172.21.15.160:0/2811674163 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec155710 con 0x7f00ec0c8f70 2024-05-09T07:14:04.367 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.363+0000 7feaaed9d640 1 -- 172.21.15.160:0/3303009754 >> v1:172.21.15.188:6789/0 conn(0x7feab0073250 legacy=0x7feab0159dd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:14:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.363+0000 7feaaed9d640 1 -- 172.21.15.160:0/3303009754 >> v1:172.21.15.160:6789/0 conn(0x7feab00c2670 legacy=0x7feab00c6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.364+0000 7f00e3fff640 1 --2- 172.21.15.160:0/2811674163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 0x7f00ec0c54b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f00d40097a0 tx=0x7f00d402a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aeb781e468934f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.363+0000 7feaaed9d640 1 -- 172.21.15.160:0/3303009754 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab015a4e0 con 0x7feab0156c90 2024-05-09T07:14:04.368 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.364+0000 7f757c476640 1 -- 172.21.15.160:0/3778409498 shutdown_connections 2024-05-09T07:14:04.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.364+0000 7f757c476640 1 -- 172.21.15.160:0/3778409498 wait complete. 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.365+0000 7f00ea7fc640 1 -- 172.21.15.160:0/2811674163 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4037070 con 0x7f00ec0c8f70 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.365+0000 7f00ea7fc640 1 -- 172.21.15.160:0/2811674163 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00d4032440 con 0x7f00ec0c8f70 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.365+0000 7f00ea7fc640 1 -- 172.21.15.160:0/2811674163 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d402ae60 con 0x7f00ec0c8f70 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 msgr2=0x7f00ec0c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f00ebfff640 1 --2- 172.21.15.160:0/2811674163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 0x7f00ec0c54b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f00d40097a0 tx=0x7f00d402a110 comp rx=0 tx=0).stop 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 shutdown_connections 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f00ebfff640 1 --2- 172.21.15.160:0/2811674163 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c8f70 0x7f00ec0c54b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f00ebfff640 1 --2- 172.21.15.160:0/2811674163 >> v2:172.21.15.188:3300/0 conn(0x7f00ec0b0350 0x7f00ec0c8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 >> 172.21.15.160:0/2811674163 conn(0x7f00ec0ac8c0 msgr2=0x7f00ec0accd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 shutdown_connections 2024-05-09T07:14:04.370 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f00ebfff640 1 -- 172.21.15.160:0/2811674163 wait complete. 2024-05-09T07:14:04.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.365+0000 7feaaed9d640 1 --2- 172.21.15.160:0/3303009754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0156c90 0x7feab015cb60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009dc0 tx=0x7feaa802a580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bb91916db58b5cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.366+0000 7f757c476640 1 Processor -- start 2024-05-09T07:14:04.371 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.365+0000 7feaadd9b640 1 -- 172.21.15.160:0/3303009754 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8007740 con 0x7feab0156c90 2024-05-09T07:14:04.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.366+0000 7f757c476640 1 -- start start 2024-05-09T07:14:04.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757c476640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f7574165c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757c476640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740c38f0 0x7f75741fada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.365+0000 7feaadd9b640 1 -- 172.21.15.160:0/3303009754 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa80078e0 con 0x7feab0156c90 2024-05-09T07:14:04.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757c476640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75740c60e0 0x7f75741f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757c476640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f757406e600 con 0x7f75740c60e0 2024-05-09T07:14:04.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757c476640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7574162740 con 0x7f75740c38f0 2024-05-09T07:14:04.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757c476640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f757406e780 con 0x7f75740c3510 2024-05-09T07:14:04.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f7579a11640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740c38f0 0x7f75741fada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f7579a11640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740c38f0 0x7f75741fada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53474/0 (socket says 172.21.15.160:53474) 2024-05-09T07:14:04.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f7579a11640 1 -- 172.21.15.160:0/4286005103 learned_addr learned my addr 172.21.15.160:0/4286005103 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7f757aa13640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75740c60e0 0x7f75741f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f757aa13640 1 -- 172.21.15.160:0/4286005103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 msgr2=0x7f7574165c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T07:14:04.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f757aa13640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f7574165c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.366+0000 7feaadd9b640 1 -- 172.21.15.160:0/3303009754 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8007cb0 con 0x7feab0156c90 2024-05-09T07:14:04.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f757aa13640 1 -- 172.21.15.160:0/4286005103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740c38f0 msgr2=0x7f75741fada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f757aa13640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740c38f0 0x7f75741fada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f757aa13640 1 -- 172.21.15.160:0/4286005103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7574160ee0 con 0x7f75740c60e0 2024-05-09T07:14:04.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.368+0000 7f757aa13640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75740c60e0 0x7f75741f8be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f756c00f180 tx=0x7f756c00f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.375 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7f00ebfff640 1 Processor -- start 2024-05-09T07:14:04.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.366+0000 7feab485c640 1 -- 172.21.15.160:0/3303009754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0156c90 msgr2=0x7feab015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.376 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7f00ebfff640 1 -- start start 2024-05-09T07:14:04.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.366+0000 7feab485c640 1 --2- 172.21.15.160:0/3303009754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0156c90 0x7feab015cb60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009dc0 tx=0x7feaa802a580 comp rx=0 tx=0).stop 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7feab485c640 1 -- 172.21.15.160:0/3303009754 shutdown_connections 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7feab485c640 1 --2- 172.21.15.160:0/3303009754 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0156c90 0x7feab015cb60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7feab485c640 1 -- 172.21.15.160:0/3303009754 >> 172.21.15.160:0/3303009754 conn(0x7feab006b370 msgr2=0x7feab006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7feab485c640 1 -- 172.21.15.160:0/3303009754 shutdown_connections 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.367+0000 7feab485c640 1 -- 172.21.15.160:0/3303009754 wait complete. 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.369+0000 7feab485c640 1 Processor -- start 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.369+0000 7feab485c640 1 -- start start 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feab485c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0073250 0x7feab00bc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feab485c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00c2670 0x7feab00be360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feab485c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 0x7feab0203ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feab485c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7feab00c32a0 con 0x7feab0156c90 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feab485c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7feab00c2fc0 con 0x7feab00c2670 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feab485c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7feab00c7510 con 0x7feab0073250 2024-05-09T07:14:04.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feaaf59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 0x7feab0203ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.369+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c00b2d0 con 0x7f75740c60e0 2024-05-09T07:14:04.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.369+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f756c01a070 con 0x7f75740c60e0 2024-05-09T07:14:04.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.369+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c009760 con 0x7f75740c60e0 2024-05-09T07:14:04.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75741fb7e0 con 0x7f75740c60e0 2024-05-09T07:14:04.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75741eb8e0 con 0x7f75740c60e0 2024-05-09T07:14:04.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feaaf59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 0x7feab0203ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44100/0 (socket says 172.21.15.160:44100) 2024-05-09T07:14:04.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7feaaf59e640 1 -- 172.21.15.160:0/3926575534 learned_addr learned my addr 172.21.15.160:0/3926575534 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.371+0000 7feaaed9d640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0073250 0x7feab00bc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.371+0000 7feaae59c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00c2670 0x7feab00be360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.371+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 0x7f00ec1eea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c59f0 0x7f00ec153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.369+0000 7f7ced619640 1 Processor -- start 2024-05-09T07:14:04.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.370+0000 7f7ced619640 1 -- start start 2024-05-09T07:14:04.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00ebfff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00ec0c8f70 0x7f00ec151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.381 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00eaffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 0x7f00ec1eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7feaaed9d640 1 -- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00c2670 msgr2=0x7feab00be360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00e3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c59f0 0x7f00ec153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00e3fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c59f0 0x7f00ec153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44416/0 (socket says 172.21.15.160:44416) 2024-05-09T07:14:04.382 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00eaffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 0x7f00ec1eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44104/0 (socket says 172.21.15.160:44104) 2024-05-09T07:14:04.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7feaaed9d640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00c2670 0x7feab00be360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00e3fff640 1 -- 172.21.15.160:0/1136892516 learned_addr learned my addr 172.21.15.160:0/1136892516 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec155fb0 con 0x7f00ec0b0350 2024-05-09T07:14:04.383 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f755c005290 con 0x7f75740c60e0 2024-05-09T07:14:04.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f00ec155cd0 con 0x7f00ec0c8f70 2024-05-09T07:14:04.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f00ec158ff0 con 0x7f00ec0c59f0 2024-05-09T07:14:04.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7feaaed9d640 1 -- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 msgr2=0x7feab0203ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.384 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7f00eb7fe640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00ec0c8f70 0x7f00ec151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7feaaed9d640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 0x7feab0203ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7feaaed9d640 1 -- 172.21.15.160:0/3926575534 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab00bbe30 con 0x7feab0073250 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f00eaffd640 1 -- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c59f0 msgr2=0x7f00ec153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f00eaffd640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c59f0 0x7f00ec153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f00eaffd640 1 -- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00ec0c8f70 msgr2=0x7f00ec151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f00eaffd640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00ec0c8f70 0x7f00ec151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.385 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f00eaffd640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00ec1eefb0 con 0x7f00ec0b0350 2024-05-09T07:14:04.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.372+0000 7feaaf59e640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 0x7feab0203ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:04.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7feaaed9d640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0073250 0x7feab00bc7e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feaa8031000 tx=0x7feaa8004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8040070 con 0x7feab0073250 2024-05-09T07:14:04.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f00eaffd640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 0x7f00ec1eea70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009fd0 tx=0x7f00dc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f7ced619640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7ce81651a0 0x7f7ce806d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f7ced619640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 0x7f7ce80c1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f7ced619640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7ce80c56a0 con 0x7f7ce808b150 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f7ced619640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f7ce80c12d0 con 0x7f7ce81651a0 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f7ced619640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce80a5fc0 con 0x7f7ce806e580 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab00bc110 con 0x7feab0073250 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7feab02039a0 con 0x7feab0073250 2024-05-09T07:14:04.387 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.373+0000 7f7ce77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 0x7f7ce80c1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feaa8031c60 con 0x7feab0073250 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce77fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 0x7f7ce80c1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44424/0 (socket says 172.21.15.160:44424) 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce77fe640 1 -- 172.21.15.160:0/3878399783 learned_addr learned my addr 172.21.15.160:0/3878399783 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce7fff640 1 --2- 172.21.15.160:0/3878399783 >> v2:172.21.15.160:3300/0 conn(0x7f7ce81651a0 0x7f7ce806d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce77fe640 1 -- 172.21.15.160:0/3878399783 >> v2:172.21.15.160:3300/0 conn(0x7f7ce81651a0 msgr2=0x7f7ce806d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce77fe640 1 --2- 172.21.15.160:0/3878399783 >> v2:172.21.15.160:3300/0 conn(0x7f7ce81651a0 0x7f7ce806d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce77fe640 1 -- 172.21.15.160:0/3878399783 >> v1:172.21.15.160:6789/0 conn(0x7f7ce808b150 legacy=0x7f7ce80c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f7ce77fe640 1 -- 172.21.15.160:0/3878399783 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce80a62a0 con 0x7f7ce806e580 2024-05-09T07:14:04.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7f3ffb093640 1 Processor -- start 2024-05-09T07:14:04.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.374+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8003bb0 con 0x7feab0073250 2024-05-09T07:14:04.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.375+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f756c005d10 con 0x7f75740c60e0 2024-05-09T07:14:04.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.375+0000 7f61bc99d640 1 Processor -- start 2024-05-09T07:14:04.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.375+0000 7f61bc99d640 1 -- start start 2024-05-09T07:14:04.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.375+0000 7f756affd640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f755806bf00 0x7f755806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.376+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f756c08b700 con 0x7f75740c60e0 2024-05-09T07:14:04.390 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.376+0000 7f3ffb093640 1 -- start start 2024-05-09T07:14:04.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.376+0000 7f757a212640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f755806bf00 0x7f755806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.376+0000 7f61bc99d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61b004a980 0x7f61b0006b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.376+0000 7f7ce77fe640 1 --2- 172.21.15.160:0/3878399783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 0x7f7ce80c1760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009e50 tx=0x7f7ce002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5f448f5f9f6b71f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.376+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea90005290 con 0x7feab0073250 2024-05-09T07:14:04.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f7ce6ffd640 1 -- 172.21.15.160:0/3878399783 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce0036070 con 0x7f7ce806e580 2024-05-09T07:14:04.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61b6d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61b004a980 0x7f61b0006b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61b6d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61b004a980 0x7f61b0006b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44116/0 (socket says 172.21.15.160:44116) 2024-05-09T07:14:04.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61b6d9d640 1 -- 172.21.15.160:0/3420628201 learned_addr learned my addr 172.21.15.160:0/3420628201 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f7ce6ffd640 1 -- 172.21.15.160:0/3878399783 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ce0031600 con 0x7f7ce806e580 2024-05-09T07:14:04.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61bc99d640 1 --2- 172.21.15.160:0/3420628201 >> v2:172.21.15.188:3300/0 conn(0x7f61b00fdeb0 0x7f61b00f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f61b00072c0 con 0x7f61b00f7590 2024-05-09T07:14:04.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f61b0030ad0 con 0x7f61b004a980 2024-05-09T07:14:04.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f61b0030db0 con 0x7f61b00fdeb0 2024-05-09T07:14:04.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ffb093640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3ff40c89a0 0x7f3ff41591e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ffb093640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3ff40cae50 0x7f3ff40cb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ffb093640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ff414f790 con 0x7f3ff414f3b0 2024-05-09T07:14:04.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ffb093640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff40c8e50 con 0x7f3ff40c89a0 2024-05-09T07:14:04.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ffb093640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3ff406d450 con 0x7f3ff40cae50 2024-05-09T07:14:04.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ff9890640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3ff40c89a0 0x7f3ff41591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ff9890640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3ff40c89a0 0x7f3ff41591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44122/0 (socket says 172.21.15.160:44122) 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.377+0000 7f3ff9890640 1 -- 172.21.15.160:0/1870909131 learned_addr learned my addr 172.21.15.160:0/1870909131 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f61b5d9b640 1 -- 172.21.15.160:0/3420628201 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 720775654 0 0) 0x7f61b00072c0 con 0x7f61b00f7590 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f61b5d9b640 1 -- 172.21.15.160:0/3420628201 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f61a0003520 con 0x7f61b00f7590 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f61b6d9d640 1 -- 172.21.15.160:0/3420628201 >> v2:172.21.15.188:3300/0 conn(0x7f61b00fdeb0 msgr2=0x7f61b00f79b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f61b6d9d640 1 --2- 172.21.15.160:0/3420628201 >> v2:172.21.15.188:3300/0 conn(0x7f61b00fdeb0 0x7f61b00f79b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f61b6d9d640 1 -- 172.21.15.160:0/3420628201 >> v1:172.21.15.188:6789/0 conn(0x7f61b00f7590 legacy=0x7f61b00fd7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f61b6d9d640 1 -- 172.21.15.160:0/3420628201 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b0031090 con 0x7f61b004a980 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f3ff9890640 1 -- 172.21.15.160:0/1870909131 >> v2:172.21.15.188:3300/0 conn(0x7f3ff40cae50 msgr2=0x7f3ff40cb280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f3ff9890640 1 --2- 172.21.15.160:0/1870909131 >> v2:172.21.15.188:3300/0 conn(0x7f3ff40cae50 0x7f3ff40cb280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f3ff9890640 1 -- 172.21.15.160:0/1870909131 >> v1:172.21.15.188:6789/0 conn(0x7f3ff414f3b0 legacy=0x7f3ff4155d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f3ff9890640 1 -- 172.21.15.160:0/1870909131 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40c5800 con 0x7f3ff40c89a0 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 -- 172.21.15.160:0/3878399783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 msgr2=0x7f7ce80c1760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 --2- 172.21.15.160:0/3878399783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 0x7f7ce80c1760 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0009e50 tx=0x7f7ce002a5f0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 -- 172.21.15.160:0/3878399783 shutdown_connections 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 --2- 172.21.15.160:0/3878399783 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce806e580 0x7f7ce80c1760 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 --2- 172.21.15.160:0/3878399783 >> v2:172.21.15.160:3300/0 conn(0x7f7ce81651a0 0x7f7ce806d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 -- 172.21.15.160:0/3878399783 >> 172.21.15.160:0/3878399783 conn(0x7f7ce80c68d0 msgr2=0x7f7ce80c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 -- 172.21.15.160:0/3878399783 shutdown_connections 2024-05-09T07:14:04.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7f7ced619640 1 -- 172.21.15.160:0/3878399783 wait complete. 2024-05-09T07:14:04.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.378+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7feaa8039040 con 0x7feab0073250 2024-05-09T07:14:04.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.379+0000 7fea977fe640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fea9c06bf30 0x7fea9c06e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.379+0000 7f61b6d9d640 1 --2- 172.21.15.160:0/3420628201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b0006b70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61b002b0a0 tx=0x7f61a802a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a42a987940b068 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.379+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7feaa80ac2a0 con 0x7feab0073250 2024-05-09T07:14:04.397 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.379+0000 7feaae59c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fea9c06bf30 0x7fea9c06e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.379+0000 7f3ff9890640 1 --2- 172.21.15.160:0/1870909131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40c89a0 0x7f3ff41591e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40c4d30 tx=0x7f3fec02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3529d2c6c4f37c5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.380+0000 7feaae59c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fea9c06bf30 0x7fea9c06e3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006fd0 tx=0x7feaa4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.398 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.380+0000 7f757a212640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f755806bf00 0x7f755806e3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757003c000 tx=0x7f757000a000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.381+0000 7f7ced619640 1 Processor -- start 2024-05-09T07:14:04.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.381+0000 7f7ced619640 1 -- start start 2024-05-09T07:14:04.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc015020 con 0x7f00ec0b0350 2024-05-09T07:14:04.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f756c059de0 con 0x7f75740c60e0 2024-05-09T07:14:04.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f61b5d9b640 1 -- 172.21.15.160:0/3420628201 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a8037070 con 0x7f61b004a980 2024-05-09T07:14:04.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f7ced619640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 0x7f7ce815ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f3ff908f640 1 -- 172.21.15.160:0/1870909131 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec036070 con 0x7f3ff40c89a0 2024-05-09T07:14:04.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00ec0c1da0 con 0x7f00ec0b0350 2024-05-09T07:14:04.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00ec0c2040 con 0x7f00ec0b0350 2024-05-09T07:14:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f3ff908f640 1 -- 172.21.15.160:0/1870909131 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3fec031600 con 0x7f3ff40c89a0 2024-05-09T07:14:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.382+0000 7f3ff908f640 1 -- 172.21.15.160:0/1870909131 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fec0319d0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00dc011830 con 0x7f00ec0b0350 2024-05-09T07:14:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc011c20 con 0x7f00ec0b0350 2024-05-09T07:14:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f61b5d9b640 1 -- 172.21.15.160:0/3420628201 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61a8032640 con 0x7f61b004a980 2024-05-09T07:14:04.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f61b5d9b640 1 -- 172.21.15.160:0/3420628201 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a80329f0 con 0x7f61b004a980 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f3ffb093640 1 -- 172.21.15.160:0/1870909131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40c89a0 msgr2=0x7f3ff41591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f3ffb093640 1 --2- 172.21.15.160:0/1870909131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40c89a0 0x7f3ff41591e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40c4d30 tx=0x7f3fec02a5f0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f3ffb093640 1 -- 172.21.15.160:0/1870909131 shutdown_connections 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f3ffb093640 1 --2- 172.21.15.160:0/1870909131 >> v2:172.21.15.188:3300/0 conn(0x7f3ff40cae50 0x7f3ff40cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f3ffb093640 1 --2- 172.21.15.160:0/1870909131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40c89a0 0x7f3ff41591e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f3ffb093640 1 -- 172.21.15.160:0/1870909131 >> 172.21.15.160:0/1870909131 conn(0x7f3ff4072a70 msgr2=0x7f3ff4072e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f3ffb093640 1 -- 172.21.15.160:0/1870909131 shutdown_connections 2024-05-09T07:14:04.402 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f3ffb093640 1 -- 172.21.15.160:0/1870909131 wait complete. 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ced619640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ce81651a0 0x7f7ce80729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 0x7f7ce815ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ce7fff640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 0x7f7ce815ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53514/0 (socket says 172.21.15.160:53514) 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ce7fff640 1 -- 172.21.15.160:0/1267056969 learned_addr learned my addr 172.21.15.160:0/1267056969 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ced619640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce815fae0 0x7f7ce808a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce80c1e20 con 0x7f7ce81651a0 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7ce80c1ca0 con 0x7f7ce808b150 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f7ce80c5270 con 0x7f7ce815fae0 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.383+0000 7f7cece18640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce815fae0 0x7f7ce808a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce7fff640 1 -- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce815fae0 msgr2=0x7f7ce808a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce7fff640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce815fae0 0x7f7ce808a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce7fff640 1 -- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ce81651a0 msgr2=0x7f7ce80729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce7fff640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ce81651a0 0x7f7ce80729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce7fff640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ce81604e0 con 0x7f7ce808b150 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce7fff640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 0x7f7ce815ffa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800d830 tx=0x7f7cd800dcf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 msgr2=0x7f61b0006b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 --2- 172.21.15.160:0/3420628201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b0006b70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f61b002b0a0 tx=0x7f61a802a330 comp rx=0 tx=0).stop 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 shutdown_connections 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 --2- 172.21.15.160:0/3420628201 >> v2:172.21.15.188:3300/0 conn(0x7f61b00fdeb0 0x7f61b00f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 --2- 172.21.15.160:0/3420628201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b0006b70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 >> 172.21.15.160:0/3420628201 conn(0x7f61b0005e90 msgr2=0x7f61b00062a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.385+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 shutdown_connections 2024-05-09T07:14:04.404 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.386+0000 7f61bc99d640 1 -- 172.21.15.160:0/3420628201 wait complete. 2024-05-09T07:14:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8018020 con 0x7f7ce808b150 2024-05-09T07:14:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7cd8009640 con 0x7f7ce808b150 2024-05-09T07:14:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ce8072f30 con 0x7f7ce808b150 2024-05-09T07:14:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.384+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ce815ef10 con 0x7f7ce808b150 2024-05-09T07:14:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.386+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd8009a10 con 0x7f7ce808b150 2024-05-09T07:14:04.405 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc8005290 con 0x7f7ce808b150 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f61bc99d640 1 Processor -- start 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f61bc99d640 1 -- start start 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.386+0000 7f3ffb093640 1 Processor -- start 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.386+0000 7f3ffb093640 1 -- start start 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffb093640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ff40c89a0 0x7f3ff41df110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffb093640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 0x7f3ff41df650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffb093640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ff414f3b0 0x7f3ff40ab3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffa091640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ff40c89a0 0x7f3ff41df110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffb093640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff4158cc0 con 0x7f3ff40cae50 2024-05-09T07:14:04.406 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffb093640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3ff41589e0 con 0x7f3ff414f3b0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffb093640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3ff40c5dc0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ff9890640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 0x7f3ff41df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ff9890640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 0x7f3ff41df650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44140/0 (socket says 172.21.15.160:44140) 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ff9890640 1 -- 172.21.15.160:0/2376271695 learned_addr learned my addr 172.21.15.160:0/2376271695 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7f3ffa892640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ff414f3b0 0x7f3ff40ab3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ffa091640 1 -- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ff414f3b0 msgr2=0x7f3ff40ab3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ffa091640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ff414f3b0 0x7f3ff40ab3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ffa091640 1 -- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 msgr2=0x7f3ff41df650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ffa091640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 0x7f3ff41df650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ffa091640 1 -- 172.21.15.160:0/2376271695 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ff40c49d0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ff9890640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 0x7f3ff41df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3ffa091640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ff40c89a0 0x7f3ff41df110 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009fd0 tx=0x7f3ff000b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0015020 con 0x7f3ff40c89a0 2024-05-09T07:14:04.407 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ff0011830 con 0x7f3ff40c89a0 2024-05-09T07:14:04.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0011c20 con 0x7f3ff40c89a0 2024-05-09T07:14:04.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.387+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7feaa807a980 con 0x7feab0073250 2024-05-09T07:14:04.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.388+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f00dc018020 con 0x7f00ec0b0350 2024-05-09T07:14:04.408 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.389+0000 7f61bc99d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b00f21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61bc99d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b00f7590 0x7f61b0192c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.389+0000 7f00e37fe640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f00cc06c030 0x7f00cc06e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.389+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f00dc0119e0 con 0x7f00ec0b0350 2024-05-09T07:14:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.389+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d0005290 con 0x7f00ec0b0350 2024-05-09T07:14:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.389+0000 7f00e3fff640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f00cc06c030 0x7f00cc06e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61b6d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b00f21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61b6d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b00f21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44148/0 (socket says 172.21.15.160:44148) 2024-05-09T07:14:04.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61b6d9d640 1 -- 172.21.15.160:0/1867064675 learned_addr learned my addr 172.21.15.160:0/1867064675 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f00e3fff640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f00cc06c030 0x7f00cc06e4f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00ec1e1330 tx=0x7f00d402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.411 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61b00fdeb0 0x7f61b00efd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61b659c640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b00f7590 0x7f61b0192c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f61b00315b0 con 0x7f61b004a980 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.390+0000 7f61b759e640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61b00fdeb0 0x7f61b00efd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.391+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61b002e080 con 0x7f61b00f7590 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.391+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61b00f0be0 con 0x7f61b00fdeb0 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.391+0000 7f61b659c640 1 -- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61b00fdeb0 msgr2=0x7f61b00efd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b659c640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61b00fdeb0 0x7f61b00efd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b659c640 1 -- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 msgr2=0x7f61b00f21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b659c640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b00f21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b659c640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61b00428a0 con 0x7f61b00f7590 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b6d9d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b00f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b759e640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61b00fdeb0 0x7f61b00efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:04.412 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f61b659c640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b00f7590 0x7f61b0192c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61b8050ef0 tx=0x7f61b8068d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7cd801b020 con 0x7f7ce808b150 2024-05-09T07:14:04.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f7ce4ff9640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7cd406beb0 0x7f7cd406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8073590 con 0x7f61b00f7590 2024-05-09T07:14:04.413 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.392+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7cd808b8e0 con 0x7f7ce808b150 2024-05-09T07:14:04.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61b807e030 con 0x7f61b00f7590 2024-05-09T07:14:04.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61b0042b80 con 0x7f61b00f7590 2024-05-09T07:14:04.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61b00f2730 con 0x7f61b00f7590 2024-05-09T07:14:04.414 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f7ce77fe640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7cd406beb0 0x7f7cd406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b806f400 con 0x7f61b00f7590 2024-05-09T07:14:04.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ff41e15a0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.393+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ff41dfd00 con 0x7f3ff40c89a0 2024-05-09T07:14:04.415 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.394+0000 7f7ce77fe640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7cd406beb0 0x7f7cd406e370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0033000 tx=0x7f7ce002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.394+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7cd8022030 con 0x7f7ce808b150 2024-05-09T07:14:04.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.395+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6198005290 con 0x7f61b00f7590 2024-05-09T07:14:04.416 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.397+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f00dc01f020 con 0x7f00ec0b0350 2024-05-09T07:14:04.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.398+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ff4067a10 con 0x7f3ff40c89a0 2024-05-09T07:14:04.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.399+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3ff0018020 con 0x7f3ff40c89a0 2024-05-09T07:14:04.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.400+0000 7f3feaffd640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3fd806bf00 0x7f3fd806e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.400+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3ff008b2e0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.401+0000 7f3ff9890640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3fd806bf00 0x7f3fd806e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.417 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.402+0000 7f3ff9890640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3fd806bf00 0x7f3fd806e3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40ab090 tx=0x7f3fec009000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.403+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61b806f5a0 con 0x7f61b00f7590 2024-05-09T07:14:04.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.403+0000 7f619f7fe640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61a406bf00 0x7f61a406e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.404+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3ff001f020 con 0x7f3ff40c89a0 2024-05-09T07:14:04.418 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.404+0000 7f61b6d9d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61a406bf00 0x7f61a406e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.405+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f61b80f4ba0 con 0x7f61b00f7590 2024-05-09T07:14:04.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.405+0000 7f61b6d9d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61a406bf00 0x7f61a406e3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009630 tx=0x7f61a802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.406+0000 7f5909438640 1 Processor -- start 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.407+0000 7f5909438640 1 -- start start 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f5909438640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59040c4fe0 0x7f59041652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f5909438640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59040c6520 con 0x7f590406dea0 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f5909438640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5904159b90 con 0x7f590406e280 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f5909438640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f590406e710 con 0x7f59040c4fe0 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f59037fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59040c4fe0 0x7f59041652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f59037fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f59040c4fe0 0x7f59041652a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53542/0 (socket says 172.21.15.160:53542) 2024-05-09T07:14:04.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f59037fe640 1 -- 172.21.15.160:0/1603981072 learned_addr learned my addr 172.21.15.160:0/1603981072 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61b80c2b10 con 0x7f61b00f7590 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f59037fe640 1 -- 172.21.15.160:0/1603981072 >> v1:172.21.15.188:6789/0 conn(0x7f590406e280 legacy=0x7f59040c48d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f59037fe640 1 -- 172.21.15.160:0/1603981072 >> v1:172.21.15.160:6789/0 conn(0x7f590406dea0 legacy=0x7f59040c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f59037fe640 1 -- 172.21.15.160:0/1603981072 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5904159e70 con 0x7f59040c4fe0 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.410+0000 7f59037fe640 1 --2- 172.21.15.160:0/1603981072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59040c4fe0 0x7f59041652a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009e50 tx=0x7f58fc02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=780a3af96f247bff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.411+0000 7f5901ffb640 1 -- 172.21.15.160:0/1603981072 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc036070 con 0x7f59040c4fe0 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.411+0000 7f5901ffb640 1 -- 172.21.15.160:0/1603981072 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58fc031600 con 0x7f59040c4fe0 2024-05-09T07:14:04.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.411+0000 7f5901ffb640 1 -- 172.21.15.160:0/1603981072 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0319d0 con 0x7f59040c4fe0 2024-05-09T07:14:04.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.412+0000 7f5909438640 1 -- 172.21.15.160:0/1603981072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59040c4fe0 msgr2=0x7f59041652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.412+0000 7f5909438640 1 --2- 172.21.15.160:0/1603981072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59040c4fe0 0x7f59041652a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f58fc009e50 tx=0x7f58fc02a710 comp rx=0 tx=0).stop 2024-05-09T07:14:04.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.413+0000 7f5909438640 1 -- 172.21.15.160:0/1603981072 shutdown_connections 2024-05-09T07:14:04.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.413+0000 7f5909438640 1 --2- 172.21.15.160:0/1603981072 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f59040c4fe0 0x7f59041652a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.413+0000 7f5909438640 1 -- 172.21.15.160:0/1603981072 >> 172.21.15.160:0/1603981072 conn(0x7f59040c7580 msgr2=0x7f59040a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.413+0000 7f5909438640 1 -- 172.21.15.160:0/1603981072 shutdown_connections 2024-05-09T07:14:04.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.406+0000 7f52bcef5640 1 Processor -- start 2024-05-09T07:14:04.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.406+0000 7f52bcef5640 1 -- start start 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52bcef5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c3c60 0x7f52b8156c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52bcef5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f52b80c1f90 0x7f52b8157140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52bcef5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52b8072ad0 0x7f52b8159c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52bcef5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f52b815dd60 con 0x7f52b8072ad0 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52bcef5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f52b815d1e0 con 0x7f52b80c1f90 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52bcef5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52b815d4c0 con 0x7f52b80c3c60 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52b6d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52b8072ad0 0x7f52b8159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52b6d9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f52b8072ad0 0x7f52b8159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44164/0 (socket says 172.21.15.160:44164) 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52b6d9d640 1 -- 172.21.15.160:0/3845140205 learned_addr learned my addr 172.21.15.160:0/3845140205 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.408+0000 7f52b5d9b640 1 --2- 172.21.15.160:0/3845140205 >> v2:172.21.15.188:3300/0 conn(0x7f52b80c1f90 0x7f52b8157140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f52b6d9d640 1 -- 172.21.15.160:0/3845140205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c3c60 msgr2=0x7f52b8156c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T07:14:04.425 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f52b6d9d640 1 --2- 172.21.15.160:0/3845140205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c3c60 0x7f52b8156c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.413+0000 7f5909438640 1 -- 172.21.15.160:0/1603981072 wait complete. 2024-05-09T07:14:04.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.415+0000 7f5909438640 1 Processor -- start 2024-05-09T07:14:04.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f52b6d9d640 1 -- 172.21.15.160:0/3845140205 >> v2:172.21.15.188:3300/0 conn(0x7f52b80c1f90 msgr2=0x7f52b8157140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:04.426 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f52b6d9d640 1 --2- 172.21.15.160:0/3845140205 >> v2:172.21.15.188:3300/0 conn(0x7f52b80c1f90 0x7f52b8157140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.415+0000 7f5909438640 1 -- start start 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.409+0000 7f52b6d9d640 1 -- 172.21.15.160:0/3845140205 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b815d7a0 con 0x7f52b8072ad0 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.410+0000 7f52b6d9d640 1 --2- 172.21.15.160:0/3845140205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 0x7f52b8159c10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f52b80716b0 tx=0x7f52b002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1837076728b274e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.411+0000 7f52b559a640 1 -- 172.21.15.160:0/3845140205 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b0058070 con 0x7f52b8072ad0 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.411+0000 7f52b559a640 1 -- 172.21.15.160:0/3845140205 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52b0053610 con 0x7f52b8072ad0 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.411+0000 7f52b559a640 1 -- 172.21.15.160:0/3845140205 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b00539e0 con 0x7f52b8072ad0 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.412+0000 7f52bcef5640 1 -- 172.21.15.160:0/3845140205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 msgr2=0x7f52b8159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.412+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3845140205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 0x7f52b8159c10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f52b80716b0 tx=0x7f52b002a5b0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f52bcef5640 1 -- 172.21.15.160:0/3845140205 shutdown_connections 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3845140205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 0x7f52b8159c10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3845140205 >> v2:172.21.15.188:3300/0 conn(0x7f52b80c1f90 0x7f52b8157140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3845140205 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c3c60 0x7f52b8156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f52bcef5640 1 -- 172.21.15.160:0/3845140205 >> 172.21.15.160:0/3845140205 conn(0x7f52b80a98a0 msgr2=0x7f52b80a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f5909438640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 0x7f5904070de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f5909438640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590406e280 0x7f59040a7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f5909438640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59040c4fe0 0x7f59041eb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 0x7f5904070de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f5902ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 0x7f5904070de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53552/0 (socket says 172.21.15.160:53552) 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.417+0000 7f5902ffd640 1 -- 172.21.15.160:0/316198858 learned_addr learned my addr 172.21.15.160:0/316198858 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f59037fe640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59040c4fe0 0x7f59041eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f590415a430 con 0x7f590406e280 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f590415a7a0 con 0x7f590406dea0 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5904157360 con 0x7f59040c4fe0 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f5902ffd640 1 -- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59040c4fe0 msgr2=0x7f59041eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f5902ffd640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59040c4fe0 0x7f59041eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f5902ffd640 1 -- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590406e280 msgr2=0x7f59040a7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.418+0000 7f59027fc640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590406e280 0x7f59040a7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.419+0000 7f5902ffd640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590406e280 0x7f59040a7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.419+0000 7f5902ffd640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5904154e80 con 0x7f590406dea0 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.419+0000 7f59027fc640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590406e280 0x7f59040a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:04.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.419+0000 7f5902ffd640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 0x7f5904070de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c410 tx=0x7f58f400c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.420+0000 7f52bcef5640 1 -- 172.21.15.160:0/3845140205 shutdown_connections 2024-05-09T07:14:04.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.420+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4017070 con 0x7f590406dea0 2024-05-09T07:14:04.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.420+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59040a84e0 con 0x7f590406dea0 2024-05-09T07:14:04.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.420+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59041ee2c0 con 0x7f590406dea0 2024-05-09T07:14:04.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.420+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f40035f0 con 0x7f590406dea0 2024-05-09T07:14:04.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.420+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4003a20 con 0x7f590406dea0 2024-05-09T07:14:04.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.422+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f58f401a070 con 0x7f590406dea0 2024-05-09T07:14:04.433 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.424+0000 7f52bcef5640 1 -- 172.21.15.160:0/3845140205 wait complete. 2024-05-09T07:14:04.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.424+0000 7f58eb7fe640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f58f006bf00 0x7f58f006e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.424+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58f408a510 con 0x7f590406dea0 2024-05-09T07:14:04.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.424+0000 7f59027fc640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f58f006bf00 0x7f58f006e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.425+0000 7f59027fc640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f58f006bf00 0x7f58f006e3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59041eddd0 tx=0x7f58f8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.427+0000 7f52bcef5640 1 Processor -- start 2024-05-09T07:14:04.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.427+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58e4005290 con 0x7f590406dea0 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.428+0000 7f52bcef5640 1 -- start start 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52bcef5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 0x7f52b80c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52bcef5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 0x7f52b81502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52bcef5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b80c3c60 0x7f52b80c5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52bcef5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f52b815cfa0 con 0x7f52b8072ad0 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52bcef5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f52b815ce20 con 0x7f52b80c3c60 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52bcef5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80c4040 con 0x7f52b80c1f90 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52b5d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 0x7f52b81502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52b5d9b640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 0x7f52b81502a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44478/0 (socket says 172.21.15.160:44478) 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.429+0000 7f52b5d9b640 1 -- 172.21.15.160:0/3090784185 learned_addr learned my addr 172.21.15.160:0/3090784185 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.430+0000 7f52b5d9b640 1 -- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b80c3c60 msgr2=0x7f52b80c5c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.430+0000 7f52b5d9b640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b80c3c60 0x7f52b80c5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.430+0000 7f52b5d9b640 1 -- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 msgr2=0x7f52b80c5310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.430+0000 7f52b5d9b640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 0x7f52b80c5310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.430+0000 7f52b5d9b640 1 -- 172.21.15.160:0/3090784185 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81507e0 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.430+0000 7f52b5d9b640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 0x7f52b81502a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e980 tx=0x7f52ac00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.431+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac017070 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.431+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b80c7230 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.431+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b80c4df0 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.431+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52ac00ca00 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.431+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00cdf0 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.433+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f52ac01a070 con 0x7f52b80c1f90 2024-05-09T07:14:04.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.433+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f58f40585a0 con 0x7f590406dea0 2024-05-09T07:14:04.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.436+0000 7f52a6ffd640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f529806bf30 0x7f529806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:04.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.436+0000 7f52b659c640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f529806bf30 0x7f529806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:04.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.437+0000 7f52b659c640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f529806bf30 0x7f529806e3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52b806f970 tx=0x7f52a8006bd0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:04.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.438+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f52ac08b0a0 con 0x7f52b80c1f90 2024-05-09T07:14:04.440 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.439+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f529c005290 con 0x7f52b80c1f90 2024-05-09T07:14:04.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.447+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f52b81507e0 con 0x7f52b80c1f90 2024-05-09T07:14:04.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.800+0000 7f7ced619640 1 -- 172.21.15.160:0/1267056969 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f7cc8005850 con 0x7f7ce808b150 2024-05-09T07:14:04.802 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.797+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fea90005580 con 0x7feab0073250 2024-05-09T07:14:04.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.801+0000 7fea977fe640 1 -- 172.21.15.160:0/3926575534 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feaa807a280 con 0x7feab0073250 2024-05-09T07:14:04.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.801+0000 7f7ce4ff9640 1 -- 172.21.15.160:0/1267056969 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cd8059fc0 con 0x7f7ce808b150 2024-05-09T07:14:04.803 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:04.808 INFO:teuthology.orchestra.run.smithi160.stdout:55834574856 2024-05-09T07:14:04.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7f7cce7fc640 1 -- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7cd406beb0 msgr2=0x7f7cd406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7f7cce7fc640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7cd406beb0 0x7f7cd406e370 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7ce0033000 tx=0x7f7ce002e000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7f7cce7fc640 1 -- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 msgr2=0x7f7ce815ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7f7cce7fc640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 0x7f7ce815ffa0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7cd800d830 tx=0x7f7cd800dcf0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fea9c06bf30 msgr2=0x7fea9c06e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7feab485c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fea9c06bf30 0x7fea9c06e3f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feaa4006fd0 tx=0x7feaa4008040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0073250 msgr2=0x7feab00bc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.807+0000 7feab485c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0073250 0x7feab00bc7e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feaa8031000 tx=0x7feaa8004a40 comp rx=0 tx=0).stop 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 -- 172.21.15.160:0/1267056969 shutdown_connections 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f7cd406beb0 0x7f7cd406e370 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7ce815fae0 0x7f7ce808a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ce81651a0 0x7f7ce80729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 --2- 172.21.15.160:0/1267056969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7ce808b150 0x7f7ce815ffa0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 -- 172.21.15.160:0/1267056969 >> 172.21.15.160:0/1267056969 conn(0x7f7ce80c68d0 msgr2=0x7f7ce81629c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 -- 172.21.15.160:0/1267056969 shutdown_connections 2024-05-09T07:14:04.811 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.809+0000 7f7cce7fc640 1 -- 172.21.15.160:0/1267056969 wait complete. 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 shutdown_connections 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fea9c06bf30 0x7fea9c06e3f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7feab00c2670 0x7feab00be360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7feab0156c90 0x7feab0203ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 --2- 172.21.15.160:0/3926575534 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7feab0073250 0x7feab00bc7e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 >> 172.21.15.160:0/3926575534 conn(0x7feab006b370 msgr2=0x7feab0159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 shutdown_connections 2024-05-09T07:14:04.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.814+0000 7feab485c640 1 -- 172.21.15.160:0/3926575534 wait complete. 2024-05-09T07:14:04.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.815+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f755c005580 con 0x7f75740c60e0 2024-05-09T07:14:04.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.815+0000 7f00ebfff640 1 -- 172.21.15.160:0/1136892516 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f00d0005580 con 0x7f00ec0b0350 2024-05-09T07:14:04.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.816+0000 7f756affd640 1 -- 172.21.15.160:0/4286005103 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f756c0596e0 con 0x7f75740c60e0 2024-05-09T07:14:04.818 INFO:teuthology.orchestra.run.smithi160.stdout:55834574856 2024-05-09T07:14:04.818 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.816+0000 7f00e37fe640 1 -- 172.21.15.160:0/1136892516 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f00ec1eefb0 con 0x7f00ec0b0350 2024-05-09T07:14:04.818 INFO:teuthology.orchestra.run.smithi160.stdout:55834574856 2024-05-09T07:14:04.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 -- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f00cc06c030 msgr2=0x7f00cc06e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f00cc06c030 0x7f00cc06e4f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00ec1e1330 tx=0x7f00d402e000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 -- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 msgr2=0x7f00ec1eea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 0x7f00ec1eea70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009fd0 tx=0x7f00dc00b870 comp rx=0 tx=0).stop 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 -- 172.21.15.160:0/1136892516 shutdown_connections 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f00cc06c030 0x7f00cc06e4f0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f00ec1e1330 tx=0x7f00d402e000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.821+0000 7f00e17fa640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f00ec0c8f70 0x7f00ec151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f00e17fa640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f00ec0c59f0 0x7f00ec153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f00e17fa640 1 --2- 172.21.15.160:0/1136892516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00ec0b0350 0x7f00ec1eea70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f00e17fa640 1 -- 172.21.15.160:0/1136892516 >> 172.21.15.160:0/1136892516 conn(0x7f00ec0ac8c0 msgr2=0x7f00ec0acca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f755806bf00 msgr2=0x7f755806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f755806bf00 0x7f755806e3c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757003c000 tx=0x7f757000a000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75740c60e0 msgr2=0x7f75741f8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75740c60e0 0x7f75741f8be0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f756c00f180 tx=0x7f756c00f640 comp rx=0 tx=0).stop 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 shutdown_connections 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f755806bf00 0x7f755806e3c0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f757003c000 tx=0x7f757000a000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f75740c38f0 0x7f75741fada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f75740c60e0 0x7f75741f8be0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 --2- 172.21.15.160:0/4286005103 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f75740c3510 0x7f7574165c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 >> 172.21.15.160:0/4286005103 conn(0x7f75740c68d0 msgr2=0x7f7574160650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.825 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f00e17fa640 1 -- 172.21.15.160:0/1136892516 shutdown_connections 2024-05-09T07:14:04.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.822+0000 7f00e17fa640 1 -- 172.21.15.160:0/1136892516 wait complete. 2024-05-09T07:14:04.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.825+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 shutdown_connections 2024-05-09T07:14:04.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.825+0000 7f757c476640 1 -- 172.21.15.160:0/4286005103 wait complete. 2024-05-09T07:14:04.831 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.829+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f6198005580 con 0x7f61b00f7590 2024-05-09T07:14:04.832 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574857 for osd.1 2024-05-09T07:14:04.836 INFO:teuthology.orchestra.run.smithi160.stdout:55834574855 2024-05-09T07:14:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.830+0000 7f619f7fe640 1 -- 172.21.15.160:0/1867064675 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f61b80c6e10 con 0x7f61b00f7590 2024-05-09T07:14:04.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.2 2024-05-09T07:14:04.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.834+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61a406bf00 msgr2=0x7f61a406e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.834+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61a406bf00 0x7f61a406e3c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f61a8009630 tx=0x7f61a802e000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.834+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b00f7590 msgr2=0x7f61b0192c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.834+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b00f7590 0x7f61b0192c80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f61b8050ef0 tx=0x7f61b8068d70 comp rx=0 tx=0).stop 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.834+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 shutdown_connections 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61a406bf00 0x7f61a406e3c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61b00f7590 0x7f61b0192c80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61b00fdeb0 0x7f61b00efd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 --2- 172.21.15.160:0/1867064675 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61b004a980 0x7f61b00f21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 >> 172.21.15.160:0/1867064675 conn(0x7f61b0005e90 msgr2=0x7f61b00495d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 shutdown_connections 2024-05-09T07:14:04.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.835+0000 7f61bc99d640 1 -- 172.21.15.160:0/1867064675 wait complete. 2024-05-09T07:14:04.842 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.6 2024-05-09T07:14:04.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.845+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f58e4005580 con 0x7f590406dea0 2024-05-09T07:14:04.847 INFO:teuthology.orchestra.run.smithi160.stdout:55834574855 2024-05-09T07:14:04.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.846+0000 7f58eb7fe640 1 -- 172.21.15.160:0/316198858 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f58f405c8a0 con 0x7f590406dea0 2024-05-09T07:14:04.850 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.4 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f58f006bf00 msgr2=0x7f58f006e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f58f006bf00 0x7f58f006e3c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f59041eddd0 tx=0x7f58f8008040 comp rx=0 tx=0).stop 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 msgr2=0x7f5904070de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 0x7f5904070de0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f58f400c410 tx=0x7f58f400c8d0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 shutdown_connections 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f58f006bf00 0x7f58f006e3c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f59040c4fe0 0x7f59041eb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f590406e280 0x7f59040a7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 --2- 172.21.15.160:0/316198858 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f590406dea0 0x7f5904070de0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 >> 172.21.15.160:0/316198858 conn(0x7f59040c7580 msgr2=0x7f59041580b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 shutdown_connections 2024-05-09T07:14:04.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.848+0000 7f5909438640 1 -- 172.21.15.160:0/316198858 wait complete. 2024-05-09T07:14:04.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.5 2024-05-09T07:14:04.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.0 2024-05-09T07:14:04.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.881+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f3ff40756c0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.881+0000 7f3feaffd640 1 -- 172.21.15.160:0/2376271695 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3ff00599c0 con 0x7f3ff40c89a0 2024-05-09T07:14:04.883 INFO:teuthology.orchestra.run.smithi160.stdout:55834574856 2024-05-09T07:14:04.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3fd806bf00 msgr2=0x7f3fd806e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3fd806bf00 0x7f3fd806e3c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40ab090 tx=0x7f3fec009000 comp rx=0 tx=0).stop 2024-05-09T07:14:04.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ff40c89a0 msgr2=0x7f3ff41df110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ff40c89a0 0x7f3ff41df110 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3ff0009fd0 tx=0x7f3ff000b870 comp rx=0 tx=0).stop 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 shutdown_connections 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3fd806bf00 0x7f3fd806e3c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3ff414f3b0 0x7f3ff40ab3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3ff40cae50 0x7f3ff41df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 --2- 172.21.15.160:0/2376271695 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3ff40c89a0 0x7f3ff41df110 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.883+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 >> 172.21.15.160:0/2376271695 conn(0x7f3ff4072a70 msgr2=0x7f3ff4154850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.884+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 shutdown_connections 2024-05-09T07:14:04.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.884+0000 7f3ffb093640 1 -- 172.21.15.160:0/2376271695 wait complete. 2024-05-09T07:14:04.896 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574856 for osd.3 2024-05-09T07:14:04.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.927+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f529c005580 con 0x7f52b80c1f90 2024-05-09T07:14:04.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.928+0000 7f52a6ffd640 1 -- 172.21.15.160:0/3090784185 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f52ac059780 con 0x7f52b80c1f90 2024-05-09T07:14:04.930 INFO:teuthology.orchestra.run.smithi160.stdout:55834574855 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f529806bf30 msgr2=0x7f529806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f529806bf30 0x7f529806e3f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52b806f970 tx=0x7f52a8006bd0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 msgr2=0x7f52b81502a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 0x7f52b81502a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e980 tx=0x7f52ac00ee40 comp rx=0 tx=0).stop 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 shutdown_connections 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f529806bf30 0x7f529806e3f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f52b80c3c60 0x7f52b80c5c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f52b80c1f90 0x7f52b81502a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 --2- 172.21.15.160:0/3090784185 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f52b8072ad0 0x7f52b80c5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 >> 172.21.15.160:0/3090784185 conn(0x7f52b80a98a0 msgr2=0x7f52b80ac740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:04.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 shutdown_connections 2024-05-09T07:14:04.933 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:04.930+0000 7f52bcef5640 1 -- 172.21.15.160:0/3090784185 wait complete. 2024-05-09T07:14:04.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.7 2024-05-09T07:14:05.833 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-09T07:14:05.837 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-09T07:14:05.843 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-09T07:14:05.851 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-09T07:14:05.853 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-09T07:14:05.862 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-09T07:14:05.897 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-09T07:14:05.944 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-09T07:14:05.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.973+0000 7fa85d8ba640 1 Processor -- start 2024-05-09T07:14:05.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.973+0000 7fa85d8ba640 1 -- start start 2024-05-09T07:14:05.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.975+0000 7fa85d8ba640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa8580c4fe0 0x7fa8581652a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:05.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.975+0000 7fa85d8ba640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8580c6520 con 0x7fa85806dea0 2024-05-09T07:14:05.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.975+0000 7fa85d8ba640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa858159b90 con 0x7fa85806e280 2024-05-09T07:14:05.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.975+0000 7fa85d8ba640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fa85806e710 con 0x7fa8580c4fe0 2024-05-09T07:14:05.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.975+0000 7fa856ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa85806dea0 0x7fa8580c5e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36382/0 (socket says 172.21.15.160:36382) 2024-05-09T07:14:05.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.975+0000 7fa856ffd640 1 -- 172.21.15.160:0/2273287891 learned_addr learned my addr 172.21.15.160:0/2273287891 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:05.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.976+0000 7fa8577fe640 1 --2- 172.21.15.160:0/2273287891 >> v2:172.21.15.160:3300/0 conn(0x7fa8580c4fe0 0x7fa8581652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:05.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.976+0000 7fa8577fe640 1 -- 172.21.15.160:0/2273287891 >> v1:172.21.15.188:6789/0 conn(0x7fa85806e280 legacy=0x7fa8580c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:05.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.976+0000 7fa8577fe640 1 -- 172.21.15.160:0/2273287891 >> v1:172.21.15.160:6789/0 conn(0x7fa85806dea0 legacy=0x7fa8580c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:05.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.976+0000 7fa8577fe640 1 -- 172.21.15.160:0/2273287891 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858159e70 con 0x7fa8580c4fe0 2024-05-09T07:14:05.979 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.978+0000 7fa8577fe640 1 --2- 172.21.15.160:0/2273287891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8580c4fe0 0x7fa8581652a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6ec3fc584efec89 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:05.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.978+0000 7fa855ffb640 1 -- 172.21.15.160:0/2273287891 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840037070 con 0x7fa8580c4fe0 2024-05-09T07:14:05.980 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.978+0000 7fa855ffb640 1 -- 172.21.15.160:0/2273287891 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84002ad60 con 0x7fa8580c4fe0 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.979+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2273287891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8580c4fe0 msgr2=0x7fa8581652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.979+0000 7fa855ffb640 1 -- 172.21.15.160:0/2273287891 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa840032660 con 0x7fa8580c4fe0 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.979+0000 7fa85d8ba640 1 --2- 172.21.15.160:0/2273287891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8580c4fe0 0x7fa8581652a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa8400097a0 tx=0x7fa84002a0d0 comp rx=0 tx=0).stop 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.979+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2273287891 shutdown_connections 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.979+0000 7fa85d8ba640 1 --2- 172.21.15.160:0/2273287891 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8580c4fe0 0x7fa8581652a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.980+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2273287891 >> 172.21.15.160:0/2273287891 conn(0x7fa8580c7580 msgr2=0x7fa8580a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:05.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.980+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2273287891 shutdown_connections 2024-05-09T07:14:05.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.980+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2273287891 wait complete. 2024-05-09T07:14:05.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.982+0000 7fa85d8ba640 1 Processor -- start 2024-05-09T07:14:05.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.982+0000 7fa85d8ba640 1 -- start start 2024-05-09T07:14:05.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.984+0000 7fa85d8ba640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 0x7fa858151530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:05.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.984+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 0x7fa858151530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:05.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.984+0000 7fa856ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 0x7fa858151530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44188/0 (socket says 172.21.15.160:44188) 2024-05-09T07:14:05.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.984+0000 7fa85d8ba640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85806e280 0x7fa858151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:05.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa85d8ba640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8580c4fe0 0x7fa858072830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:05.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa85d8ba640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa85815a430 con 0x7fa85806dea0 2024-05-09T07:14:05.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa85d8ba640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa85815a7a0 con 0x7fa85806e280 2024-05-09T07:14:05.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa85d8ba640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa858157360 con 0x7fa8580c4fe0 2024-05-09T07:14:05.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa856ffd640 1 -- 172.21.15.160:0/2215956853 learned_addr learned my addr 172.21.15.160:0/2215956853 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:05.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa856ffd640 1 -- 172.21.15.160:0/2215956853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8580c4fe0 msgr2=0x7fa858072830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:05.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa856ffd640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8580c4fe0 0x7fa858072830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:05.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa856ffd640 1 -- 172.21.15.160:0/2215956853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85806e280 msgr2=0x7fa858151a70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T07:14:05.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa856ffd640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85806e280 0x7fa858151a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:05.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.985+0000 7fa856ffd640 1 -- 172.21.15.160:0/2215956853 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa858070960 con 0x7fa85806dea0 2024-05-09T07:14:05.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.986+0000 7fa856ffd640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 0x7fa858151530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c8f0 tx=0x7fa85000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:05.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.987+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa850018070 con 0x7fa85806dea0 2024-05-09T07:14:05.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.987+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8500072d0 con 0x7fa85806dea0 2024-05-09T07:14:05.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.987+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2215956853 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa858070c00 con 0x7fa85806dea0 2024-05-09T07:14:05.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.990+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2215956853 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa858070ee0 con 0x7fa85806dea0 2024-05-09T07:14:05.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.990+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8500047e0 con 0x7fa85806dea0 2024-05-09T07:14:05.992 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.990+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa85001b070 con 0x7fa85806dea0 2024-05-09T07:14:05.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.991+0000 7fa84ffff640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa83806bf60 0x7fa83806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:05.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.991+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa85008a470 con 0x7fa85806dea0 2024-05-09T07:14:05.996 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.995+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2215956853 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83c005290 con 0x7fa85806dea0 2024-05-09T07:14:06.000 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:05.999+0000 7fa8567fc640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa83806bf60 0x7fa83806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.001 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.000+0000 7fa8567fc640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa83806bf60 0x7fa83806e420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8581507a0 tx=0x7fa848007d30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.006 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.004+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa850058bb0 con 0x7fa85806dea0 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.019+0000 7f9dab9fb640 1 Processor -- start 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.019+0000 7f9dab9fb640 1 -- start start 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.020+0000 7f9dab9fb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da406b160 0x7f9da406b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.020+0000 7f9dab9fb640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9da40c12d0 0x7f9da40c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.020+0000 7f9dab9fb640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9da415cf30 0x7f9da415d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.020+0000 7f9dab9fb640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f9da415a5f0 con 0x7f9da40c12d0 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.020+0000 7f9dab9fb640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9da40c62c0 con 0x7f9da415cf30 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.020+0000 7f9dab9fb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9da41599e0 con 0x7f9da406b160 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.021+0000 7f9da9797640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da406b160 0x7f9da406b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.021+0000 7f9da9f98640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9da415cf30 0x7f9da415d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9da40c12d0 0x7f9da40c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9da40c12d0 0x7f9da40c5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44190/0 (socket says 172.21.15.160:44190) 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 -- 172.21.15.160:0/2395405731 learned_addr learned my addr 172.21.15.160:0/2395405731 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 -- 172.21.15.160:0/2395405731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da406b160 msgr2=0x7f9da406b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 --2- 172.21.15.160:0/2395405731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da406b160 0x7f9da406b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 -- 172.21.15.160:0/2395405731 >> v2:172.21.15.188:3300/0 conn(0x7f9da415cf30 msgr2=0x7f9da415d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 --2- 172.21.15.160:0/2395405731 >> v2:172.21.15.188:3300/0 conn(0x7f9da415cf30 0x7f9da415d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.022+0000 7f9da8f96640 1 -- 172.21.15.160:0/2395405731 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da4159cc0 con 0x7f9da40c12d0 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.023+0000 7f9da8f96640 1 --2- 172.21.15.160:0/2395405731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da40c12d0 0x7f9da40c5d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9da40c11a0 tx=0x7f9d9c02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7d79f49baeb442 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.024+0000 7f9d9bfff640 1 -- 172.21.15.160:0/2395405731 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c036070 con 0x7f9da40c12d0 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.024+0000 7f9d9bfff640 1 -- 172.21.15.160:0/2395405731 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d9c031600 con 0x7f9da40c12d0 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.024+0000 7f9d9bfff640 1 -- 172.21.15.160:0/2395405731 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c02ae60 con 0x7f9da40c12d0 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 -- 172.21.15.160:0/2395405731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da40c12d0 msgr2=0x7f9da40c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.029 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 --2- 172.21.15.160:0/2395405731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da40c12d0 0x7f9da40c5d80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9da40c11a0 tx=0x7f9d9c02a640 comp rx=0 tx=0).stop 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 -- 172.21.15.160:0/2395405731 shutdown_connections 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 --2- 172.21.15.160:0/2395405731 >> v2:172.21.15.188:3300/0 conn(0x7f9da415cf30 0x7f9da415d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 --2- 172.21.15.160:0/2395405731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da40c12d0 0x7f9da40c5d80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 --2- 172.21.15.160:0/2395405731 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da406b160 0x7f9da406b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 -- 172.21.15.160:0/2395405731 >> 172.21.15.160:0/2395405731 conn(0x7f9da40c7150 msgr2=0x7f9da40a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 -- 172.21.15.160:0/2395405731 shutdown_connections 2024-05-09T07:14:06.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.025+0000 7f9dab9fb640 1 -- 172.21.15.160:0/2395405731 wait complete. 2024-05-09T07:14:06.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.028+0000 7f9dab9fb640 1 Processor -- start 2024-05-09T07:14:06.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.028+0000 7f9dab9fb640 1 -- start start 2024-05-09T07:14:06.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9dab9fb640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da406b160 0x7f9da40aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9dab9fb640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da415cf30 0x7f9da40ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9dab9fb640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da4158340 0x7f9da41587c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9dab9fb640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9da406c7b0 con 0x7f9da406b160 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9dab9fb640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9da415a280 con 0x7f9da415cf30 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9dab9fb640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9da415d8f0 con 0x7f9da4158340 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.029+0000 7f9da9797640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da406b160 0x7f9da40aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da8f96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da415cf30 0x7f9da40ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da8f96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da415cf30 0x7f9da40ab4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53590/0 (socket says 172.21.15.160:53590) 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da8f96640 1 -- 172.21.15.160:0/3969463649 learned_addr learned my addr 172.21.15.160:0/3969463649 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da9f98640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da4158340 0x7f9da41587c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da9f98640 1 -- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da415cf30 msgr2=0x7f9da40ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da9f98640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da415cf30 0x7f9da40ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da9f98640 1 -- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da406b160 msgr2=0x7f9da40aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da9f98640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da406b160 0x7f9da40aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.030+0000 7f9da9f98640 1 -- 172.21.15.160:0/3969463649 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9da4159650 con 0x7f9da4158340 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.031+0000 7f9da9797640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da406b160 0x7f9da40aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:06.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.031+0000 7f9da9f98640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da4158340 0x7f9da41587c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c4f0 tx=0x7f9d9400c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.032+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94015070 con 0x7f9da4158340 2024-05-09T07:14:06.036 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.033+0000 7f9dab9fb640 1 -- 172.21.15.160:0/3969463649 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9da4158d00 con 0x7f9da4158340 2024-05-09T07:14:06.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.033+0000 7f9dab9fb640 1 -- 172.21.15.160:0/3969463649 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9da4157b40 con 0x7f9da4158340 2024-05-09T07:14:06.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.033+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d94004240 con 0x7f9da4158340 2024-05-09T07:14:06.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.033+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d94003610 con 0x7f9da4158340 2024-05-09T07:14:06.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.035+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9d940037b0 con 0x7f9da4158340 2024-05-09T07:14:06.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.036+0000 7f9dab9fb640 1 -- 172.21.15.160:0/3969463649 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d84005290 con 0x7f9da4158340 2024-05-09T07:14:06.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.036+0000 7f6a6d9f9640 1 Processor -- start 2024-05-09T07:14:06.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.036+0000 7f9d99ffb640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9d8806bf30 0x7f9d8806e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.037+0000 7f9da9797640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9d8806bf30 0x7f9d8806e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.037+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9d9408acc0 con 0x7f9da4158340 2024-05-09T07:14:06.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.037+0000 7f6a6d9f9640 1 -- start start 2024-05-09T07:14:06.039 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.037+0000 7f826ca3c640 1 Processor -- start 2024-05-09T07:14:06.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.037+0000 7f826ca3c640 1 -- start start 2024-05-09T07:14:06.040 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.038+0000 7f6a6d9f9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a680a8d90 0x7f6a6806e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.038+0000 7f6a6d9f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f6a680c62c0 0x7f6a6814e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.038+0000 7f6a6d9f9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6a680c6700 con 0x7f6a6806b160 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.038+0000 7f6a6d9f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f6a6806c670 con 0x7f6a680c62c0 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.038+0000 7f6a6d9f9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f6a6806cc30 con 0x7f6a680a8d90 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a680a8d90 0x7f6a6806e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f6a680a8d90 0x7f6a6806e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53606/0 (socket says 172.21.15.160:53606) 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 -- 172.21.15.160:0/2669284975 learned_addr learned my addr 172.21.15.160:0/2669284975 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.041 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f826ca3c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260040990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f826ca3c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82600df0e0 con 0x7f82600dbeb0 2024-05-09T07:14:06.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f826ca3c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82600df3c0 con 0x7f8260040ed0 2024-05-09T07:14:06.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f826ca3c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f826000be40 con 0x7f826000d7b0 2024-05-09T07:14:06.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f8267fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f8260040ed0 0x7f826000c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43318/0 (socket says 172.21.15.160:43318) 2024-05-09T07:14:06.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f9da9797640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9d8806bf30 0x7f9d8806e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9da40a8970 tx=0x7f9da0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.042 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f8267fff640 1 -- 172.21.15.160:0/1365096176 learned_addr learned my addr 172.21.15.160:0/1365096176 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:06.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f8266ffd640 1 --2- 172.21.15.160:0/1365096176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260040990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 -- 172.21.15.160:0/2669284975 >> v2:172.21.15.160:3300/0 conn(0x7f6a680c62c0 msgr2=0x7f6a6814e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 --2- 172.21.15.160:0/2669284975 >> v2:172.21.15.160:3300/0 conn(0x7f6a680c62c0 0x7f6a6814e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 -- 172.21.15.160:0/2669284975 >> v1:172.21.15.160:6789/0 conn(0x7f6a6806b160 legacy=0x7f6a680c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.043 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.039+0000 7f6a66ffd640 1 -- 172.21.15.160:0/2669284975 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a681522d0 con 0x7f6a680a8d90 2024-05-09T07:14:06.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.040+0000 7fe97df4e640 1 Processor -- start 2024-05-09T07:14:06.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.040+0000 7fe97df4e640 1 -- start start 2024-05-09T07:14:06.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.040+0000 7f8266ffd640 1 -- 172.21.15.160:0/1365096176 >> v1:172.21.15.188:6789/0 conn(0x7f8260040ed0 legacy=0x7f826000c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.040+0000 7f8266ffd640 1 -- 172.21.15.160:0/1365096176 >> v1:172.21.15.160:6789/0 conn(0x7f82600dbeb0 legacy=0x7f82600de900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.044 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.040+0000 7f8266ffd640 1 -- 172.21.15.160:0/1365096176 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82600df6a0 con 0x7f826000d7b0 2024-05-09T07:14:06.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7f6a66ffd640 1 --2- 172.21.15.160:0/2669284975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a680a8d90 0x7f6a6806e2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a680a9440 tx=0x7f6a6002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269e80d645bf6ac5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7f6a65ffb640 1 -- 172.21.15.160:0/2669284975 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60036070 con 0x7f6a680a8d90 2024-05-09T07:14:06.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7f6a65ffb640 1 -- 172.21.15.160:0/2669284975 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60031600 con 0x7f6a680a8d90 2024-05-09T07:14:06.045 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7f6a65ffb640 1 -- 172.21.15.160:0/2669284975 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a600319d0 con 0x7f6a680a8d90 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7fe97df4e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe9700d7920 0x7fe97000edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7fe97df4e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 0x7fe9700062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7fe97df4e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9700d7d00 con 0x7fe97000fcf0 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7fe97df4e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe9700db250 con 0x7fe9700d7920 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7fe97df4e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe97004ba00 con 0x7fe970050720 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7fe977fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 0x7fe9700062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.046 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.041+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9d94058d50 con 0x7f9da4158340 2024-05-09T07:14:06.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7fe977fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 0x7fe9700062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44522/0 (socket says 172.21.15.160:44522) 2024-05-09T07:14:06.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7fe977fff640 1 -- 172.21.15.160:0/4081461395 learned_addr learned my addr 172.21.15.160:0/4081461395 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f8266ffd640 1 --2- 172.21.15.160:0/1365096176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260040990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82540097d0 tx=0x7f825402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18da4ef6e583938b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.047 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7fe97cf4c640 1 --2- 172.21.15.160:0/4081461395 >> v2:172.21.15.160:3300/0 conn(0x7fe9700d7920 0x7fe97000edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/2669284975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a680a8d90 msgr2=0x7f6a6806e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/2669284975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a680a8d90 0x7f6a6806e2c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6a680a9440 tx=0x7f6a6002a660 comp rx=0 tx=0).stop 2024-05-09T07:14:06.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f82667fc640 1 -- 172.21.15.160:0/1365096176 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254037070 con 0x7f826000d7b0 2024-05-09T07:14:06.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/2669284975 shutdown_connections 2024-05-09T07:14:06.048 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/2669284975 >> v2:172.21.15.160:3300/0 conn(0x7f6a680c62c0 0x7f6a6814e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/2669284975 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a680a8d90 0x7f6a6806e2c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/2669284975 >> 172.21.15.160:0/2669284975 conn(0x7f6a680c7150 msgr2=0x7f6a680a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/2669284975 shutdown_connections 2024-05-09T07:14:06.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/2669284975 wait complete. 2024-05-09T07:14:06.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7fe977fff640 1 -- 172.21.15.160:0/4081461395 >> v2:172.21.15.160:3300/0 conn(0x7fe9700d7920 msgr2=0x7fe97000edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.049 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7fe9777fe640 1 -- 172.21.15.160:0/4081461395 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1103229688 0 0) 0x7fe9700d7d00 con 0x7fe97000fcf0 2024-05-09T07:14:06.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.042+0000 7f82667fc640 1 -- 172.21.15.160:0/1365096176 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f825402ae40 con 0x7f826000d7b0 2024-05-09T07:14:06.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7f826ca3c640 1 -- 172.21.15.160:0/1365096176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 msgr2=0x7f8260040990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7f82667fc640 1 -- 172.21.15.160:0/1365096176 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254037070 con 0x7f826000d7b0 2024-05-09T07:14:06.050 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7f826ca3c640 1 --2- 172.21.15.160:0/1365096176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260040990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f82540097d0 tx=0x7f825402a0f0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7fe977fff640 1 --2- 172.21.15.160:0/4081461395 >> v2:172.21.15.160:3300/0 conn(0x7fe9700d7920 0x7fe97000edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7fe977fff640 1 -- 172.21.15.160:0/4081461395 >> v1:172.21.15.188:6789/0 conn(0x7fe97000fcf0 legacy=0x7fe970006800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.043+0000 7fe977fff640 1 -- 172.21.15.160:0/4081461395 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe97004bce0 con 0x7fe970050720 2024-05-09T07:14:06.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7fe97cf4c640 1 --2- 172.21.15.160:0/4081461395 >> v2:172.21.15.160:3300/0 conn(0x7fe9700d7920 0x7fe97000edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:06.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f826ca3c640 1 -- 172.21.15.160:0/1365096176 shutdown_connections 2024-05-09T07:14:06.051 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f826ca3c640 1 --2- 172.21.15.160:0/1365096176 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260040990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f6a6d9f9640 1 Processor -- start 2024-05-09T07:14:06.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f826ca3c640 1 -- 172.21.15.160:0/1365096176 >> 172.21.15.160:0/1365096176 conn(0x7f826002f4b0 msgr2=0x7f826002f890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.052 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f826ca3c640 1 -- 172.21.15.160:0/1365096176 shutdown_connections 2024-05-09T07:14:06.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f6a6d9f9640 1 -- start start 2024-05-09T07:14:06.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7f6a6d9f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 0x7f6a680c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7f6a6d9f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a680a8d90 0x7f6a68154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7f6a6d9f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a680c62c0 0x7f6a6814bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.053 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7f6a6d9f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6814ee20 con 0x7f6a680a8d90 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7f6a6d9f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680bdb10 con 0x7f6a6806b160 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7f6a6d9f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6a6806b590 con 0x7f6a680c62c0 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.046+0000 7f6a66ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 0x7f6a680c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.046+0000 7f6a66ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 0x7f6a680c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53612/0 (socket says 172.21.15.160:53612) 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.046+0000 7f6a66ffd640 1 -- 172.21.15.160:0/640852613 learned_addr learned my addr 172.21.15.160:0/640852613 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7fe977fff640 1 --2- 172.21.15.160:0/4081461395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 0x7fe9700062c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=add93a3069946309 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7fe9777fe640 1 -- 172.21.15.160:0/4081461395 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037070 con 0x7fe970050720 2024-05-09T07:14:06.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7fe9777fe640 1 -- 172.21.15.160:0/4081461395 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964032440 con 0x7fe970050720 2024-05-09T07:14:06.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 -- 172.21.15.160:0/4081461395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 msgr2=0x7fe9700062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 --2- 172.21.15.160:0/4081461395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 0x7fe9700062c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe964009870 tx=0x7fe96402a210 comp rx=0 tx=0).stop 2024-05-09T07:14:06.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 -- 172.21.15.160:0/4081461395 shutdown_connections 2024-05-09T07:14:06.055 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 --2- 172.21.15.160:0/4081461395 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe970050720 0x7fe9700062c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 --2- 172.21.15.160:0/4081461395 >> v2:172.21.15.160:3300/0 conn(0x7fe9700d7920 0x7fe97000edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 -- 172.21.15.160:0/4081461395 >> 172.21.15.160:0/4081461395 conn(0x7fe97004fc00 msgr2=0x7fe970050010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 -- 172.21.15.160:0/4081461395 shutdown_connections 2024-05-09T07:14:06.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.045+0000 7fe97df4e640 1 -- 172.21.15.160:0/4081461395 wait complete. 2024-05-09T07:14:06.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.044+0000 7f826ca3c640 1 -- 172.21.15.160:0/1365096176 wait complete. 2024-05-09T07:14:06.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.046+0000 7f826ca3c640 1 Processor -- start 2024-05-09T07:14:06.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.046+0000 7f6a667fc640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a680a8d90 0x7f6a68154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a66ffd640 1 -- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a680c62c0 msgr2=0x7f6a6814bb40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T07:14:06.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a66ffd640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a680c62c0 0x7f6a6814bb40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a66ffd640 1 -- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a680a8d90 msgr2=0x7f6a68154cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a66ffd640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a680a8d90 0x7f6a68154cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a66ffd640 1 -- 172.21.15.160:0/640852613 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a68073160 con 0x7f6a6806b160 2024-05-09T07:14:06.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.046+0000 7f826ca3c640 1 -- start start 2024-05-09T07:14:06.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a66ffd640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 0x7f6a680c3be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a60034040 tx=0x7f6a60004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.047+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60041020 con 0x7f6a6806b160 2024-05-09T07:14:06.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7fe97df4e640 1 Processor -- start 2024-05-09T07:14:06.060 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f826ca3c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260042100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f826ca3c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 0x7f8260031760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f826ca3c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82600dbeb0 0x7f82600daa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.061 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f826ca3c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f82600dc290 con 0x7f82600dbeb0 2024-05-09T07:14:06.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a680c41e0 con 0x7f6a6806b160 2024-05-09T07:14:06.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a680c1550 con 0x7f6a6806b160 2024-05-09T07:14:06.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a60007b20 con 0x7f6a6806b160 2024-05-09T07:14:06.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7fe97df4e640 1 -- start start 2024-05-09T07:14:06.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97df4e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97000fcf0 0x7fe9700de450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97df4e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700d7920 0x7fe97004ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97df4e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 0x7fe9700ccf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97df4e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe970006f10 con 0x7fe9700d7920 2024-05-09T07:14:06.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97df4e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe97004c2a0 con 0x7fe97000fcf0 2024-05-09T07:14:06.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97df4e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700dad20 con 0x7fe9700d9e50 2024-05-09T07:14:06.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a60036040 con 0x7f6a6806b160 2024-05-09T07:14:06.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7fe97d74d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 0x7fe9700ccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe97d74d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 0x7fe9700ccf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:44546/0 (socket says 172.21.15.160:44546) 2024-05-09T07:14:06.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a48005290 con 0x7f6a6806b160 2024-05-09T07:14:06.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe97d74d640 1 -- 172.21.15.160:0/276485979 learned_addr learned my addr 172.21.15.160:0/276485979 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe977fff640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700d7920 0x7fe97004ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe97cf4c640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97000fcf0 0x7fe9700de450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f8266ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 0x7f8260031760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f8266ffd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 0x7f8260031760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53628/0 (socket says 172.21.15.160:53628) 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f8266ffd640 1 -- 172.21.15.160:0/296529835 learned_addr learned my addr 172.21.15.160:0/296529835 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f8267fff640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82600dbeb0 0x7f82600daa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f826ca3c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82600491d0 con 0x7f8260040ed0 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.048+0000 7f826ca3c640 1 -- 172.21.15.160:0/296529835 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f826000ccc0 con 0x7f826000d7b0 2024-05-09T07:14:06.066 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7f8266ffd640 1 -- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 msgr2=0x7f8260042100 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T07:14:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7f8266ffd640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260042100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7f8266ffd640 1 -- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82600dbeb0 msgr2=0x7f82600daa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7f8266ffd640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82600dbeb0 0x7f82600daa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.049+0000 7f8266ffd640 1 -- 172.21.15.160:0/296529835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8260178b30 con 0x7f8260040ed0 2024-05-09T07:14:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7f8267fff640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82600dbeb0 0x7f82600daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:06.067 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7f8266ffd640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 0x7f8260031760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8254035040 tx=0x7f82540046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe977fff640 1 -- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 msgr2=0x7fe9700ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe977fff640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 0x7fe9700ccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe977fff640 1 -- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97000fcf0 msgr2=0x7fe9700de450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe977fff640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97000fcf0 0x7fe9700de450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe977fff640 1 -- 172.21.15.160:0/276485979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700d3740 con 0x7fe9700d7920 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.050+0000 7fe97d74d640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 0x7fe9700ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:06.068 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7fe977fff640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700d7920 0x7fe97004ecc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe964031000 tx=0x7fe9640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254042020 con 0x7f8260040ed0 2024-05-09T07:14:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f6a60044020 con 0x7f6a6806b160 2024-05-09T07:14:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f6a4ffff640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6a5406be30 0x7f6a5406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f6a600ac1f0 con 0x7f6a6806b160 2024-05-09T07:14:06.069 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7f826ca3c640 1 -- 172.21.15.160:0/296529835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8260033160 con 0x7f8260040ed0 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7f826ca3c640 1 -- 172.21.15.160:0/296529835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82600db5a0 con 0x7f8260040ed0 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8254003d40 con 0x7f8260040ed0 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254037040 con 0x7f8260040ed0 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82600057e0 con 0x7f8260040ed0 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.053+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8254045020 con 0x7f8260040ed0 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f8253fff640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f824c06bf00 0x7f824c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f4bd6763640 1 Processor -- start 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.052+0000 7f4bd6763640 1 -- start start 2024-05-09T07:14:06.070 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.053+0000 7f4bd6763640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c35d0 0x7f4bd006dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f4bcf7fe640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f4bd00aa450 0x7f4bd00c2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43332/0 (socket says 172.21.15.160:43332) 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f4bd6763640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd0151790 con 0x7f4bd00c39b0 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f4bd6763640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4bd0151a50 con 0x7f4bd00aa450 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f4bd6763640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd0152380 con 0x7f4bd00c35d0 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c35d0 0x7f4bd006dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f4bcf7fe640 1 -- 172.21.15.160:0/88498129 learned_addr learned my addr 172.21.15.160:0/88498129 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964041020 con 0x7fe9700d7920 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe964030bc0 con 0x7fe9700d7920 2024-05-09T07:14:06.071 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe964037040 con 0x7fe9700d7920 2024-05-09T07:14:06.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7fe97df4e640 1 -- 172.21.15.160:0/276485979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9700d3a20 con 0x7fe9700d7920 2024-05-09T07:14:06.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.051+0000 7fe97df4e640 1 -- 172.21.15.160:0/276485979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9700d8e30 con 0x7fe9700d7920 2024-05-09T07:14:06.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe950005290 con 0x7fe9700d7920 2024-05-09T07:14:06.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f82540abc80 con 0x7f8260040ed0 2024-05-09T07:14:06.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.054+0000 7f6a667fc640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6a5406be30 0x7f6a5406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.055+0000 7f4bcffff640 1 -- 172.21.15.160:0/88498129 >> v1:172.21.15.188:6789/0 conn(0x7f4bd00aa450 legacy=0x7f4bd00c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.055+0000 7f4bceffd640 1 -- 172.21.15.160:0/88498129 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3717752601 0 0) 0x7f4bd0151790 con 0x7f4bd00c39b0 2024-05-09T07:14:06.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.055+0000 7f6a667fc640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6a5406be30 0x7f6a5406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a68155600 tx=0x7f6a58008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.055+0000 7f4bcffff640 1 -- 172.21.15.160:0/88498129 >> v1:172.21.15.160:6789/0 conn(0x7f4bd00c39b0 legacy=0x7f4bd015c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.055+0000 7f4bcffff640 1 -- 172.21.15.160:0/88498129 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd0151d30 con 0x7f4bd00c35d0 2024-05-09T07:14:06.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.056+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe964044020 con 0x7fe9700d7920 2024-05-09T07:14:06.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.056+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6a6004b030 con 0x7f6a6806b160 2024-05-09T07:14:06.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.055+0000 7f82677fe640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f824c06bf00 0x7f824c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.057+0000 7f82677fe640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f824c06bf00 0x7f824c06e3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f825c000f60 tx=0x7f825c0079b0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.057+0000 7fe9757fa640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe95c06bf00 0x7fe95c06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.057+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fe9640ab940 con 0x7fe9700d7920 2024-05-09T07:14:06.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.058+0000 7f4bcffff640 1 --2- 172.21.15.160:0/88498129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c35d0 0x7f4bd006dff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40097d0 tx=0x7f4bc402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bce65c367be3c40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.058+0000 7fe97cf4c640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe95c06bf00 0x7fe95c06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.058+0000 7f4bceffd640 1 -- 172.21.15.160:0/88498129 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4037070 con 0x7f4bd00c35d0 2024-05-09T07:14:06.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.058+0000 7f4bceffd640 1 -- 172.21.15.160:0/88498129 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4032440 con 0x7f4bd00c35d0 2024-05-09T07:14:06.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.058+0000 7f4bceffd640 1 -- 172.21.15.160:0/88498129 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4032810 con 0x7f4bd00c35d0 2024-05-09T07:14:06.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.059+0000 7fe97cf4c640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe95c06bf00 0x7fe95c06e3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00aac0 tx=0x7fe96c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.059+0000 7f4bd6763640 1 -- 172.21.15.160:0/88498129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c35d0 msgr2=0x7f4bd006dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.059+0000 7f4bd6763640 1 --2- 172.21.15.160:0/88498129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c35d0 0x7f4bd006dff0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4bc40097d0 tx=0x7f4bc402a180 comp rx=0 tx=0).stop 2024-05-09T07:14:06.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.059+0000 7f4bd6763640 1 -- 172.21.15.160:0/88498129 shutdown_connections 2024-05-09T07:14:06.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.059+0000 7f4bd6763640 1 --2- 172.21.15.160:0/88498129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c35d0 0x7f4bd006dff0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.059+0000 7f4bd6763640 1 -- 172.21.15.160:0/88498129 >> 172.21.15.160:0/88498129 conn(0x7f4bd00a98e0 msgr2=0x7f4bd00c73a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.060+0000 7f4bd6763640 1 -- 172.21.15.160:0/88498129 shutdown_connections 2024-05-09T07:14:06.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.060+0000 7f4bd6763640 1 -- 172.21.15.160:0/88498129 wait complete. 2024-05-09T07:14:06.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.061+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f82540b2080 con 0x7f8260040ed0 2024-05-09T07:14:06.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.062+0000 7f4bd6763640 1 Processor -- start 2024-05-09T07:14:06.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.062+0000 7f4bd6763640 1 -- start start 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.063+0000 7f4bd6763640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 0x7f4bd00bcc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.064+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 0x7f4bd00bcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.064+0000 7f4bcffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 0x7f4bd00bcc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44228/0 (socket says 172.21.15.160:44228) 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.064+0000 7f4bcffff640 1 -- 172.21.15.160:0/3817155829 learned_addr learned my addr 172.21.15.160:0/3817155829 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.064+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c39b0 0x7f4bd014e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.064+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd013e280 0x7f4bd014e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.065+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00c1da0 con 0x7f4bd00c35d0 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.065+0000 7f4bcf7fe640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c39b0 0x7f4bd014e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.065+0000 7f4bd4d00640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd013e280 0x7f4bd014e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.065+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4bd00c7ab0 con 0x7f4bd013e280 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.065+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4bd00c1940 con 0x7f4bd00c39b0 2024-05-09T07:14:06.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.066+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe96404b030 con 0x7fe9700d7920 2024-05-09T07:14:06.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.066+0000 7f4bcffff640 1 -- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c39b0 msgr2=0x7f4bd014e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.066+0000 7f4bcffff640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c39b0 0x7f4bd014e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.066+0000 7f4bcffff640 1 -- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd013e280 msgr2=0x7f4bd014e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.066+0000 7f4bcffff640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd013e280 0x7f4bd014e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.066+0000 7f4bcffff640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bd0139e70 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.067+0000 7f4bd4d00640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd013e280 0x7f4bd014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.067+0000 7f4bcf7fe640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c39b0 0x7f4bd014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.067+0000 7f4bcffff640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 0x7f4bd00bcc70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4bc402a770 tx=0x7f4bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.068+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4042020 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.068+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bd014c040 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.068+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bd014c5c0 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.072+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4bc4030bf0 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.073+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bc4037040 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.073+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4bc4045020 con 0x7f4bd00c35d0 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.074+0000 7f4bccff9640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4bbc06bf60 0x7f4bbc06e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.075+0000 7f4bcf7fe640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4bbc06bf60 0x7f4bbc06e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.075+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bb0005290 con 0x7f4bd00c35d0 2024-05-09T07:14:06.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.085+0000 7f4bcf7fe640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4bbc06bf60 0x7f4bbc06e420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0160710 tx=0x7f4bc0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.086+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4bc40ac0b0 con 0x7f4bd00c35d0 2024-05-09T07:14:06.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.086+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4bc404c030 con 0x7f4bd00c35d0 2024-05-09T07:14:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.103+0000 7f508755f640 1 Processor -- start 2024-05-09T07:14:06.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.104+0000 7f508755f640 1 -- start start 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.106+0000 7f508755f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50780dfd90 0x7f507803eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.106+0000 7f508755f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f50780e2550 0x7f50780e29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.106+0000 7f508755f640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50780474e0 con 0x7f507803e7a0 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.106+0000 7f508755f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5078046960 con 0x7f50780e2550 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.106+0000 7f508755f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50780477c0 con 0x7f50780dfd90 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.106+0000 7f508655d640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f507803e7a0 0x7f50780df320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43346/0 (socket says 172.21.15.160:43346) 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508655d640 1 -- 172.21.15.160:0/815165810 learned_addr learned my addr 172.21.15.160:0/815165810 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f5086d5e640 1 --2- 172.21.15.160:0/815165810 >> v2:172.21.15.188:3300/0 conn(0x7f50780e2550 0x7f50780e29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f5085d5c640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50780dfd90 0x7f507803eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.109 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3316795009 0 0) 0x7f50780474e0 con 0x7f507803e7a0 2024-05-09T07:14:06.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f506c003520 con 0x7f507803e7a0 2024-05-09T07:14:06.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1111070216 0 0) 0x7f506c003520 con 0x7f507803e7a0 2024-05-09T07:14:06.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50780dfd90 msgr2=0x7f507803eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50780dfd90 0x7f507803eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 >> v2:172.21.15.188:3300/0 conn(0x7f50780e2550 msgr2=0x7f50780e29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 --2- 172.21.15.160:0/815165810 >> v2:172.21.15.188:3300/0 conn(0x7f50780e2550 0x7f50780e29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5078047aa0 con 0x7f507803e7a0 2024-05-09T07:14:06.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f5070001670 con 0x7f507803e7a0 2024-05-09T07:14:06.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.107+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 >> v1:172.21.15.188:6789/0 conn(0x7f507803e7a0 legacy=0x7f50780df320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508555b640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c0039b0 0x7f506c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508555b640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f506c004800 0x7f506c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508555b640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0051c0 0x7f506c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5078046960 con 0x7f506c004800 2024-05-09T07:14:06.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50780477c0 con 0x7f506c0051c0 2024-05-09T07:14:06.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f506c00c060 con 0x7f506c0039b0 2024-05-09T07:14:06.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f508655d640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c0039b0 0x7f506c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.114 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.108+0000 7f5085d5c640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f506c004800 0x7f506c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f5086d5e640 1 --2- 172.21.15.160:0/815165810 >> v2:172.21.15.188:3300/0 conn(0x7f50780e2550 0x7f50780e29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:06.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f5085d5c640 1 -- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c0039b0 msgr2=0x7f506c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f5085d5c640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c0039b0 0x7f506c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f5085d5c640 1 -- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0051c0 msgr2=0x7f506c00b9c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:14:06.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f5085d5c640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f506c0051c0 0x7f506c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f5085d5c640 1 -- 172.21.15.160:0/815165810 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c003520 con 0x7f506c004800 2024-05-09T07:14:06.116 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.109+0000 7f508655d640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c0039b0 0x7f506c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:06.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.110+0000 7f5085d5c640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f506c004800 0x7f506c004c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5078047260 tx=0x7f506c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd08c8de2357c181 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.110+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c036070 con 0x7f506c004800 2024-05-09T07:14:06.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.111+0000 7f508555b640 1 -- 172.21.15.160:0/815165810 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f507c031400 con 0x7f506c004800 2024-05-09T07:14:06.117 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.111+0000 7f508755f640 1 -- 172.21.15.160:0/815165810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f506c004800 msgr2=0x7f506c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.111+0000 7f508755f640 1 --2- 172.21.15.160:0/815165810 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f506c004800 0x7f506c004c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5078047260 tx=0x7f506c005830 comp rx=0 tx=0).stop 2024-05-09T07:14:06.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.111+0000 7f5086d5e640 1 -- 172.21.15.160:0/815165810 reap_dead start 2024-05-09T07:14:06.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.112+0000 7f508755f640 1 -- 172.21.15.160:0/815165810 shutdown_connections 2024-05-09T07:14:06.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.112+0000 7f508755f640 1 -- 172.21.15.160:0/815165810 >> 172.21.15.160:0/815165810 conn(0x7f5078048250 msgr2=0x7f5078048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.119 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.112+0000 7f508755f640 1 -- 172.21.15.160:0/815165810 shutdown_connections 2024-05-09T07:14:06.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.112+0000 7f508755f640 1 -- 172.21.15.160:0/815165810 wait complete. 2024-05-09T07:14:06.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.114+0000 7f508755f640 1 Processor -- start 2024-05-09T07:14:06.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.114+0000 7f508755f640 1 -- start start 2024-05-09T07:14:06.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.115+0000 7f508755f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c004800 0x7f5078043ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.120 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.115+0000 7f508755f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50780d9de0 0x7f507818ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.115+0000 7f508755f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50780de660 0x7f50780d7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.115+0000 7f508755f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f507800b9a0 con 0x7f50780de660 2024-05-09T07:14:06.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.115+0000 7f508755f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50780e2030 con 0x7f50780d9de0 2024-05-09T07:14:06.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.115+0000 7f508755f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f507800bb20 con 0x7f506c004800 2024-05-09T07:14:06.121 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.116+0000 7f5086d5e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50780de660 0x7f50780d7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.116+0000 7f5085d5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50780d9de0 0x7f507818ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.116+0000 7f5085d5c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50780d9de0 0x7f507818ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:53672/0 (socket says 172.21.15.160:53672) 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.116+0000 7f5085d5c640 1 -- 172.21.15.160:0/1994906500 learned_addr learned my addr 172.21.15.160:0/1994906500 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.117+0000 7f5086d5e640 1 -- 172.21.15.160:0/1994906500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c004800 msgr2=0x7f5078043ce0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.117+0000 7f5086d5e640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c004800 0x7f5078043ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.117+0000 7f5086d5e640 1 -- 172.21.15.160:0/1994906500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50780d9de0 msgr2=0x7f507818ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.117+0000 7f5086d5e640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50780d9de0 0x7f507818ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.117+0000 7f5086d5e640 1 -- 172.21.15.160:0/1994906500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50780d9210 con 0x7f50780de660 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.117+0000 7f5086d5e640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50780de660 0x7f50780d7bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f50800503a0 tx=0x7f5080068d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.122 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.118+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5080068580 con 0x7f50780de660 2024-05-09T07:14:06.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.118+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50780da270 con 0x7f50780de660 2024-05-09T07:14:06.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.118+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50780d73d0 con 0x7f50780de660 2024-05-09T07:14:06.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.121+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50800734a0 con 0x7f50780de660 2024-05-09T07:14:06.123 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.121+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50800713f0 con 0x7f50780de660 2024-05-09T07:14:06.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.121+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5080071650 con 0x7f50780de660 2024-05-09T07:14:06.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.122+0000 7f5076ffd640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f506406bf30 0x7f506406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.122+0000 7f508655d640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f506406bf30 0x7f506406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.124 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.123+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5080106320 con 0x7f50780de660 2024-05-09T07:14:06.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.123+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f507802d0e0 con 0x7f50780de660 2024-05-09T07:14:06.125 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.123+0000 7f508655d640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f506406bf30 0x7f506406e3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50700079f0 tx=0x7f5070009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.132+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f508009a030 con 0x7f50780de660 2024-05-09T07:14:06.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.143+0000 7f2074aed640 1 Processor -- start 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.143+0000 7f2074aed640 1 -- start start 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f2074aed640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f2068006640 0x7f20680474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f2074aed640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f206803f030 0x7f2068007ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f2074aed640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20680df7d0 con 0x7f20680484a0 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f2074aed640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f20680dc2e0 con 0x7f206803f030 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f2074aed640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f20680df2a0 con 0x7f2068006640 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f206ffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f206803f030 0x7f2068007ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f206ffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f206803f030 0x7f2068007ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44260/0 (socket says 172.21.15.160:44260) 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f206ffff640 1 -- 172.21.15.160:0/237989220 learned_addr learned my addr 172.21.15.160:0/237989220 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.144+0000 7f206effd640 1 --2- 172.21.15.160:0/237989220 >> v2:172.21.15.188:3300/0 conn(0x7f2068006640 0x7f20680474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.145+0000 7f206effd640 1 -- 172.21.15.160:0/237989220 >> v2:172.21.15.160:3300/0 conn(0x7f206803f030 msgr2=0x7f2068007ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.145+0000 7f206effd640 1 --2- 172.21.15.160:0/237989220 >> v2:172.21.15.160:3300/0 conn(0x7f206803f030 0x7f2068007ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.145+0000 7f206effd640 1 -- 172.21.15.160:0/237989220 >> v1:172.21.15.188:6789/0 conn(0x7f20680484a0 legacy=0x7f20680e2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.145+0000 7f206effd640 1 -- 172.21.15.160:0/237989220 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20680e1f40 con 0x7f2068006640 2024-05-09T07:14:06.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.146+0000 7f206effd640 1 --2- 172.21.15.160:0/237989220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068006640 0x7f20680474d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2068009a20 tx=0x7f205c02a0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d52f932a7812b4f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.146+0000 7f206e7fc640 1 -- 172.21.15.160:0/237989220 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c037070 con 0x7f2068006640 2024-05-09T07:14:06.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.146+0000 7f206e7fc640 1 -- 172.21.15.160:0/237989220 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f205c02adf0 con 0x7f2068006640 2024-05-09T07:14:06.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.146+0000 7f206e7fc640 1 -- 172.21.15.160:0/237989220 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c032660 con 0x7f2068006640 2024-05-09T07:14:06.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 -- 172.21.15.160:0/237989220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068006640 msgr2=0x7f20680474d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 --2- 172.21.15.160:0/237989220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068006640 0x7f20680474d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2068009a20 tx=0x7f205c02a0a0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 -- 172.21.15.160:0/237989220 shutdown_connections 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 --2- 172.21.15.160:0/237989220 >> v2:172.21.15.160:3300/0 conn(0x7f206803f030 0x7f2068007ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 --2- 172.21.15.160:0/237989220 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2068006640 0x7f20680474d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 -- 172.21.15.160:0/237989220 >> 172.21.15.160:0/237989220 conn(0x7f206802db90 msgr2=0x7f206802df70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 -- 172.21.15.160:0/237989220 shutdown_connections 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.147+0000 7f2074aed640 1 -- 172.21.15.160:0/237989220 wait complete. 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.149+0000 7f2074aed640 1 Processor -- start 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.149+0000 7f2074aed640 1 -- start start 2024-05-09T07:14:06.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.150+0000 7f2074aed640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2068006640 0x7f20680c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.150+0000 7f2074aed640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206803f030 0x7f206802cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.150+0000 7f2074aed640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 0x7f20680cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.150+0000 7f2074aed640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f20680e21e0 con 0x7f20680484a0 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f2074aed640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f2068047a10 con 0x7f206803f030 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f2074aed640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f206800bd20 con 0x7f2068006640 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f206ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 0x7f20680cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f206ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 0x7f20680cbd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44268/0 (socket says 172.21.15.160:44268) 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f206ffff640 1 -- 172.21.15.160:0/3652851277 learned_addr learned my addr 172.21.15.160:0/3652851277 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f206f7fe640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2068006640 0x7f20680c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.151+0000 7f206effd640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206803f030 0x7f206802cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.152+0000 7f206f7fe640 1 -- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206803f030 msgr2=0x7f206802cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.152+0000 7f206f7fe640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206803f030 0x7f206802cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.152+0000 7f206f7fe640 1 -- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 msgr2=0x7f20680cbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.152+0000 7f206f7fe640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 0x7f20680cbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.152+0000 7f206f7fe640 1 -- 172.21.15.160:0/3652851277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20680c9bd0 con 0x7f2068006640 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.153+0000 7f206ffff640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 0x7f20680cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.153+0000 7f206f7fe640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2068006640 0x7f20680c9690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f206400c3c0 tx=0x7f206400c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.153+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20640052d0 con 0x7f2068006640 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.153+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20680cd3b0 con 0x7f2068006640 2024-05-09T07:14:06.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.153+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f20680cd970 con 0x7f2068006640 2024-05-09T07:14:06.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.153+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2064005490 con 0x7f2068006640 2024-05-09T07:14:06.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.155+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2064007260 con 0x7f2068006640 2024-05-09T07:14:06.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.156+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2064007440 con 0x7f2068006640 2024-05-09T07:14:06.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.157+0000 7f205bfff640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f205406bf30 0x7f205406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.157+0000 7f206effd640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f205406bf30 0x7f205406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.158+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2050005290 con 0x7f2068006640 2024-05-09T07:14:06.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.158+0000 7f206effd640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f205406bf30 0x7f205406e3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f205c034000 tx=0x7f205c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.158+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f206408a310 con 0x7f2068006640 2024-05-09T07:14:06.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.163+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f20640583a0 con 0x7f2068006640 2024-05-09T07:14:06.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.420+0000 7fa85d8ba640 1 -- 172.21.15.160:0/2215956853 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa83c005850 con 0x7fa85806dea0 2024-05-09T07:14:06.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.421+0000 7fa84ffff640 1 -- 172.21.15.160:0/2215956853 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa8500584b0 con 0x7fa85806dea0 2024-05-09T07:14:06.423 INFO:teuthology.orchestra.run.smithi160.stdout:55834574859 2024-05-09T07:14:06.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.425+0000 7fa84dffb640 1 -- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa83806bf60 msgr2=0x7fa83806e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.425+0000 7fa84dffb640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa83806bf60 0x7fa83806e420 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8581507a0 tx=0x7fa848007d30 comp rx=0 tx=0).stop 2024-05-09T07:14:06.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.425+0000 7fa84dffb640 1 -- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 msgr2=0x7fa858151530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.425+0000 7fa84dffb640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 0x7fa858151530 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa85000c8f0 tx=0x7fa85000cdb0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.426+0000 7fa84dffb640 1 -- 172.21.15.160:0/2215956853 shutdown_connections 2024-05-09T07:14:06.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.426+0000 7fa84dffb640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa83806bf60 0x7fa83806e420 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8581507a0 tx=0x7fa848007d30 comp rx=0 tx=0).stop 2024-05-09T07:14:06.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.426+0000 7fa84dffb640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa85806e280 0x7fa858151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.426+0000 7fa84dffb640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8580c4fe0 0x7fa858072830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.426+0000 7fa84dffb640 1 --2- 172.21.15.160:0/2215956853 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa85806dea0 0x7fa858151530 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.426+0000 7fa84dffb640 1 -- 172.21.15.160:0/2215956853 >> 172.21.15.160:0/2215956853 conn(0x7fa8580c7580 msgr2=0x7fa8580c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.427+0000 7fa84dffb640 1 -- 172.21.15.160:0/2215956853 shutdown_connections 2024-05-09T07:14:06.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.427+0000 7fa84dffb640 1 -- 172.21.15.160:0/2215956853 wait complete. 2024-05-09T07:14:06.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.428+0000 7f9dab9fb640 1 -- 172.21.15.160:0/3969463649 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f9d84005580 con 0x7f9da4158340 2024-05-09T07:14:06.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.429+0000 7f9d99ffb640 1 -- 172.21.15.160:0/3969463649 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d9405d050 con 0x7f9da4158340 2024-05-09T07:14:06.430 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:06.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 -- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9d8806bf30 msgr2=0x7f9d8806e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9d8806bf30 0x7f9d8806e3f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9da40a8970 tx=0x7f9da0008040 comp rx=0 tx=0).stop 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 -- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da4158340 msgr2=0x7f9da41587c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da4158340 0x7f9da41587c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9d9400c4f0 tx=0x7f9d9400c9b0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 -- 172.21.15.160:0/3969463649 shutdown_connections 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9d8806bf30 0x7f9d8806e3f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9da415cf30 0x7f9da40ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.433+0000 7f9d8f7fe640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9da4158340 0x7f9da41587c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.434+0000 7f9d8f7fe640 1 --2- 172.21.15.160:0/3969463649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9da406b160 0x7f9da40aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.434+0000 7f9d8f7fe640 1 -- 172.21.15.160:0/3969463649 >> 172.21.15.160:0/3969463649 conn(0x7f9da40c7150 msgr2=0x7f9da40a6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.434+0000 7f9d8f7fe640 1 -- 172.21.15.160:0/3969463649 shutdown_connections 2024-05-09T07:14:06.436 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.434+0000 7f9d8f7fe640 1 -- 172.21.15.160:0/3969463649 wait complete. 2024-05-09T07:14:06.447 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-05-09T07:14:06.448 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.4 2024-05-09T07:14:06.452 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.455+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6a48005580 con 0x7f6a6806b160 2024-05-09T07:14:06.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.456+0000 7f6a4ffff640 1 -- 172.21.15.160:0/640852613 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6a6007a850 con 0x7f6a6806b160 2024-05-09T07:14:06.458 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:06.460 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.458+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6a5406be30 msgr2=0x7f6a5406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.460 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.458+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6a5406be30 0x7f6a5406e2f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6a68155600 tx=0x7f6a58008040 comp rx=0 tx=0).stop 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.458+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 msgr2=0x7f6a680c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.458+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 0x7f6a680c3be0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f6a60034040 tx=0x7f6a60004a40 comp rx=0 tx=0).stop 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 shutdown_connections 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f6a5406be30 0x7f6a5406e2f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f6a680c62c0 0x7f6a6814bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6a680a8d90 0x7f6a68154cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 --2- 172.21.15.160:0/640852613 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f6a6806b160 0x7f6a680c3be0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 >> 172.21.15.160:0/640852613 conn(0x7f6a680c7150 msgr2=0x7f6a680a8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 shutdown_connections 2024-05-09T07:14:06.461 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.459+0000 7f6a6d9f9640 1 -- 172.21.15.160:0/640852613 wait complete. 2024-05-09T07:14:06.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.467+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8260033940 con 0x7f8260040ed0 2024-05-09T07:14:06.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.467+0000 7f8253fff640 1 -- 172.21.15.160:0/296529835 <== mon.1 v2:172.21.15.188:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f825404c030 con 0x7f8260040ed0 2024-05-09T07:14:06.469 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:06.472 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-05-09T07:14:06.472 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f824c06bf00 msgr2=0x7f824c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f824c06bf00 0x7f824c06e3c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f825c000f60 tx=0x7f825c0079b0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 msgr2=0x7f8260031760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 0x7f8260031760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8254035040 tx=0x7f82540046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 shutdown_connections 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f824c06bf00 0x7f824c06e3c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8260040ed0 0x7f8260031760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82600dbeb0 0x7f82600daa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 --2- 172.21.15.160:0/296529835 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f826000d7b0 0x7f8260042100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 >> 172.21.15.160:0/296529835 conn(0x7f826002f4b0 msgr2=0x7f826000c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 shutdown_connections 2024-05-09T07:14:06.474 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.471+0000 7f8251ffb640 1 -- 172.21.15.160:0/296529835 wait complete. 2024-05-09T07:14:06.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.486+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe950005580 con 0x7fe9700d7920 2024-05-09T07:14:06.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.6 2024-05-09T07:14:06.489 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.486+0000 7fe9757fa640 1 -- 172.21.15.160:0/276485979 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe96407a140 con 0x7fe9700d7920 2024-05-09T07:14:06.489 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.489 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:06.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.489+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe95c06bf00 msgr2=0x7fe95c06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.491 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.489+0000 7fe962ffd640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe95c06bf00 0x7fe95c06e3c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe96c00aac0 tx=0x7fe96c008040 comp rx=0 tx=0).stop 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.489+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700d7920 msgr2=0x7fe97004ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.489+0000 7fe962ffd640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700d7920 0x7fe97004ecc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe964031000 tx=0x7fe9640046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.489+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 shutdown_connections 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.489+0000 7fe962ffd640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe95c06bf00 0x7fe95c06e3c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.490+0000 7fe962ffd640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe9700d9e50 0x7fe9700ccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.490+0000 7fe962ffd640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe9700d7920 0x7fe97004ecc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.490+0000 7fe962ffd640 1 --2- 172.21.15.160:0/276485979 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe97000fcf0 0x7fe9700de450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.490+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 >> 172.21.15.160:0/276485979 conn(0x7fe97004fc00 msgr2=0x7fe9700061c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.490+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 shutdown_connections 2024-05-09T07:14:06.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.490+0000 7fe962ffd640 1 -- 172.21.15.160:0/276485979 wait complete. 2024-05-09T07:14:06.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.497+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4bb0005580 con 0x7f4bd00c35d0 2024-05-09T07:14:06.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.497+0000 7f4bccff9640 1 -- 172.21.15.160:0/3817155829 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4bd0139e70 con 0x7f4bd00c35d0 2024-05-09T07:14:06.499 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:06.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4bbc06bf60 msgr2=0x7f4bbc06e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.502 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4bbc06bf60 0x7f4bbc06e420 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4bd0160710 tx=0x7f4bc0009040 comp rx=0 tx=0).stop 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 msgr2=0x7f4bd00bcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 0x7f4bd00bcc70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4bc402a770 tx=0x7f4bc40046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 shutdown_connections 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4bbc06bf60 0x7f4bbc06e420 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4bd013e280 0x7f4bd014e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4bd00c39b0 0x7f4bd014e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 --2- 172.21.15.160:0/3817155829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4bd00c35d0 0x7f4bd00bcc70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.500+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 >> 172.21.15.160:0/3817155829 conn(0x7f4bd00a98e0 msgr2=0x7f4bd00c4f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.501+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 shutdown_connections 2024-05-09T07:14:06.503 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.501+0000 7f4bd6763640 1 -- 172.21.15.160:0/3817155829 wait complete. 2024-05-09T07:14:06.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.503+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2050005580 con 0x7f2068006640 2024-05-09T07:14:06.505 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.503+0000 7f205bfff640 1 -- 172.21.15.160:0/3652851277 <== mon.2 v2:172.21.15.188:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f206405c6a0 con 0x7f2068006640 2024-05-09T07:14:06.505 INFO:teuthology.orchestra.run.smithi160.stdout:55834574857 2024-05-09T07:14:06.507 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2024-05-09T07:14:06.507 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f205406bf30 msgr2=0x7f205406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f205406bf30 0x7f205406e3f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f205c034000 tx=0x7f205c02e000 comp rx=0 tx=0).stop 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2068006640 msgr2=0x7f20680c9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2068006640 0x7f20680c9690 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f206400c3c0 tx=0x7f206400c880 comp rx=0 tx=0).stop 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 shutdown_connections 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f205406bf30 0x7f205406e3f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f206803f030 0x7f206802cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20680484a0 0x7f20680cbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 --2- 172.21.15.160:0/3652851277 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f2068006640 0x7f20680c9690 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 >> 172.21.15.160:0/3652851277 conn(0x7f206802db90 msgr2=0x7f20680dde40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 shutdown_connections 2024-05-09T07:14:06.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.507+0000 7f2074aed640 1 -- 172.21.15.160:0/3652851277 wait complete. 2024-05-09T07:14:06.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2024-05-09T07:14:06.518 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2024-05-09T07:14:06.522 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.521+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5078007430 con 0x7f50780de660 2024-05-09T07:14:06.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.522+0000 7f5076ffd640 1 -- 172.21.15.160:0/1994906500 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f50800d4a00 con 0x7f50780de660 2024-05-09T07:14:06.524 INFO:teuthology.orchestra.run.smithi160.stdout:55834574858 2024-05-09T07:14:06.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f506406bf30 msgr2=0x7f506406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f506406bf30 0x7f506406e3f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f50700079f0 tx=0x7f5070009040 comp rx=0 tx=0).stop 2024-05-09T07:14:06.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50780de660 msgr2=0x7f50780d7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50780de660 0x7f50780d7bd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f50800503a0 tx=0x7f5080068d70 comp rx=0 tx=0).stop 2024-05-09T07:14:06.526 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 shutdown_connections 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f506406bf30 0x7f506406e3f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50780d9de0 0x7f507818ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50780de660 0x7f50780d7bd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 --2- 172.21.15.160:0/1994906500 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f506c004800 0x7f5078043ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 >> 172.21.15.160:0/1994906500 conn(0x7f5078048250 msgr2=0x7f507804afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 shutdown_connections 2024-05-09T07:14:06.527 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.524+0000 7f508755f640 1 -- 172.21.15.160:0/1994906500 wait complete. 2024-05-09T07:14:06.537 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.3 2024-05-09T07:14:06.537 DEBUG:teuthology.parallel:result is None 2024-05-09T07:14:06.537 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T07:14:06.537 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:14:06.678 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.676+0000 7f6710876640 1 Processor -- start 2024-05-09T07:14:06.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.677+0000 7f6710876640 1 -- start start 2024-05-09T07:14:06.679 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.677+0000 7f6710876640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f670c0a5fb0 0x7f670c0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.677+0000 7f6710876640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670c1563b0 con 0x7f670c0bc360 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.677+0000 7f6710876640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f670c156690 con 0x7f670c153300 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.677+0000 7f6710876640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f670c0c1630 con 0x7f670c0a5fb0 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f6709d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f670c0a5fb0 0x7f670c0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f670ad9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f670c0bc360 0x7f670c15cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36410/0 (socket says 172.21.15.160:36410) 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f670ad9d640 1 -- 172.21.15.160:0/3209061549 learned_addr learned my addr 172.21.15.160:0/3209061549 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f6709d9b640 1 -- 172.21.15.160:0/3209061549 >> v1:172.21.15.188:6789/0 conn(0x7f670c153300 legacy=0x7f670c0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f6709d9b640 1 -- 172.21.15.160:0/3209061549 >> v1:172.21.15.160:6789/0 conn(0x7f670c0bc360 legacy=0x7f670c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f670959a640 1 -- 172.21.15.160:0/3209061549 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2097521457 0 0) 0x7f670c1563b0 con 0x7f670c0bc360 2024-05-09T07:14:06.680 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.678+0000 7f6709d9b640 1 -- 172.21.15.160:0/3209061549 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c0c1c20 con 0x7f670c0a5fb0 2024-05-09T07:14:06.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6709d9b640 1 --2- 172.21.15.160:0/3209061549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c0a5fb0 0x7f670c0bbe20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009d90 tx=0x7f66f402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab797a27baa242ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f670959a640 1 -- 172.21.15.160:0/3209061549 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4007740 con 0x7f670c0a5fb0 2024-05-09T07:14:06.681 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f670959a640 1 -- 172.21.15.160:0/3209061549 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66f40078e0 con 0x7f670c0a5fb0 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f670959a640 1 -- 172.21.15.160:0/3209061549 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66f4007cb0 con 0x7f670c0a5fb0 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 -- 172.21.15.160:0/3209061549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c0a5fb0 msgr2=0x7f670c0bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 --2- 172.21.15.160:0/3209061549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c0a5fb0 0x7f670c0bbe20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f66f4009d90 tx=0x7f66f402a6a0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 -- 172.21.15.160:0/3209061549 shutdown_connections 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 --2- 172.21.15.160:0/3209061549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c0a5fb0 0x7f670c0bbe20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 -- 172.21.15.160:0/3209061549 >> 172.21.15.160:0/3209061549 conn(0x7f670c0c7580 msgr2=0x7f670c0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 -- 172.21.15.160:0/3209061549 shutdown_connections 2024-05-09T07:14:06.682 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.679+0000 7f6710876640 1 -- 172.21.15.160:0/3209061549 wait complete. 2024-05-09T07:14:06.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 Processor -- start 2024-05-09T07:14:06.683 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 -- start start 2024-05-09T07:14:06.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f670c0a5fb0 0x7f670c0aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.684 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f670c0bc360 0x7f670c14f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 0x7f670c0a9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0c2420 con 0x7f670c153300 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f670c0c19d0 con 0x7f670c0bc360 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f6710876640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f670c0c5ff0 con 0x7f670c0a5fb0 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.681+0000 7f670ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 0x7f670c0a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 0x7f670c0a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44282/0 (socket says 172.21.15.160:44282) 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 -- 172.21.15.160:0/878366863 learned_addr learned my addr 172.21.15.160:0/878366863 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670a59c640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f670c0a5fb0 0x7f670c0aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f6709d9b640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f670c0bc360 0x7f670c14f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 -- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f670c0a5fb0 msgr2=0x7f670c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f670c0a5fb0 0x7f670c0aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 -- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f670c0bc360 msgr2=0x7f670c14f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f670c0bc360 0x7f670c14f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.685 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 -- 172.21.15.160:0/878366863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c14a3b0 con 0x7f670c153300 2024-05-09T07:14:06.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f670ad9d640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 0x7f670c0a9770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f670400c450 tx=0x7f670400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704018070 con 0x7f670c153300 2024-05-09T07:14:06.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6704007500 con 0x7f670c153300 2024-05-09T07:14:06.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c1ecc70 con 0x7f670c153300 2024-05-09T07:14:06.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704004a20 con 0x7f670c153300 2024-05-09T07:14:06.686 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.682+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c0ab710 con 0x7f670c153300 2024-05-09T07:14:06.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.683+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c0756c0 con 0x7f670c153300 2024-05-09T07:14:06.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.683+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f670401b070 con 0x7f670c153300 2024-05-09T07:14:06.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.684+0000 7f6702ffd640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f66ec06be60 0x7f66ec06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:06.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.684+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f670408a3d0 con 0x7f670c153300 2024-05-09T07:14:06.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.684+0000 7f670a59c640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f66ec06be60 0x7f66ec06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:06.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.684+0000 7f670a59c640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f66ec06be60 0x7f66ec06e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66fc004ea0 tx=0x7f66fc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:06.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.688+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6704058ad0 con 0x7f670c153300 2024-05-09T07:14:06.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.858+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f670c0c17b0 con 0x7f66ec06be60 2024-05-09T07:14:06.867 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.865+0000 7f6702ffd640 1 -- 172.21.15.160:0/878366863 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f670c0c17b0 con 0x7f66ec06be60 2024-05-09T07:14:06.871 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:06.872 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:14:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f66ec06be60 msgr2=0x7f66ec06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f66ec06be60 0x7f66ec06e320 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f66fc004ea0 tx=0x7f66fc009040 comp rx=0 tx=0).stop 2024-05-09T07:14:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 msgr2=0x7f670c0a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 0x7f670c0a9770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f670400c450 tx=0x7f670400c910 comp rx=0 tx=0).stop 2024-05-09T07:14:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 shutdown_connections 2024-05-09T07:14:06.874 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f66ec06be60 0x7f66ec06e320 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f670c0bc360 0x7f670c14f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f670c153300 0x7f670c0a9770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 --2- 172.21.15.160:0/878366863 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f670c0a5fb0 0x7f670c0aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 >> 172.21.15.160:0/878366863 conn(0x7f670c0c7580 msgr2=0x7f670c0a8660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 shutdown_connections 2024-05-09T07:14:06.875 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:06.869+0000 7f6710876640 1 -- 172.21.15.160:0/878366863 wait complete. 2024-05-09T07:14:06.882 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-05-09T07:14:05.182654+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002289"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199324+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:13:52.199324+0000","last_peered":"2024-05-09T07:13:52.199324+0000","last_clean":"2024-05-09T07:13:52.199324+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:13:52.199324+0000","last_undegraded":"2024-05-09T07:13:52.199324+0000","last_fullsized":"2024-05-09T07:13:52.199324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491364+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:13:51.491364+0000","last_peered":"2024-05-09T07:13:51.491364+0000","last_clean":"2024-05-09T07:13:51.491364+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:13:51.491364+0000","last_undegraded":"2024-05-09T07:13:51.491364+0000","last_fullsized":"2024-05-09T07:13:51.491364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490779+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:13:51.490779+0000","last_peered":"2024-05-09T07:13:51.490779+0000","last_clean":"2024-05-09T07:13:51.490779+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:13:51.490779+0000","last_undegraded":"2024-05-09T07:13:51.490779+0000","last_fullsized":"2024-05-09T07:13:51.490779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491562+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:13:51.491562+0000","last_peered":"2024-05-09T07:13:51.491562+0000","last_clean":"2024-05-09T07:13:51.491562+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:13:51.491562+0000","last_undegraded":"2024-05-09T07:13:51.491562+0000","last_fullsized":"2024-05-09T07:13:51.491562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199187+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:13:52.199187+0000","last_peered":"2024-05-09T07:13:52.199187+0000","last_clean":"2024-05-09T07:13:52.199187+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:13:52.199187+0000","last_undegraded":"2024-05-09T07:13:52.199187+0000","last_fullsized":"2024-05-09T07:13:52.199187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800899+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:13:51.800899+0000","last_peered":"2024-05-09T07:13:51.800899+0000","last_clean":"2024-05-09T07:13:51.800899+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:13:51.800899+0000","last_undegraded":"2024-05-09T07:13:51.800899+0000","last_fullsized":"2024-05-09T07:13:51.800899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698480+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:13:51.698480+0000","last_peered":"2024-05-09T07:13:51.698480+0000","last_clean":"2024-05-09T07:13:51.698480+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:13:51.698480+0000","last_undegraded":"2024-05-09T07:13:51.698480+0000","last_fullsized":"2024-05-09T07:13:51.698480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202587+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:13:52.202587+0000","last_peered":"2024-05-09T07:13:52.202587+0000","last_clean":"2024-05-09T07:13:52.202587+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:13:52.202587+0000","last_undegraded":"2024-05-09T07:13:52.202587+0000","last_fullsized":"2024-05-09T07:13:52.202587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801107+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:13:51.801107+0000","last_peered":"2024-05-09T07:13:51.801107+0000","last_clean":"2024-05-09T07:13:51.801107+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:13:51.801107+0000","last_undegraded":"2024-05-09T07:13:51.801107+0000","last_fullsized":"2024-05-09T07:13:51.801107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698221+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:13:51.698221+0000","last_peered":"2024-05-09T07:13:51.698221+0000","last_clean":"2024-05-09T07:13:51.698221+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:13:51.698221+0000","last_undegraded":"2024-05-09T07:13:51.698221+0000","last_fullsized":"2024-05-09T07:13:51.698221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202703+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:13:52.202703+0000","last_peered":"2024-05-09T07:13:52.202703+0000","last_clean":"2024-05-09T07:13:52.202703+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:13:52.202703+0000","last_undegraded":"2024-05-09T07:13:52.202703+0000","last_fullsized":"2024-05-09T07:13:52.202703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800950+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:13:51.800950+0000","last_peered":"2024-05-09T07:13:51.800950+0000","last_clean":"2024-05-09T07:13:51.800950+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:13:51.800950+0000","last_undegraded":"2024-05-09T07:13:51.800950+0000","last_fullsized":"2024-05-09T07:13:51.800950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201596+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:13:52.201596+0000","last_peered":"2024-05-09T07:13:52.201596+0000","last_clean":"2024-05-09T07:13:52.201596+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:13:52.201596+0000","last_undegraded":"2024-05-09T07:13:52.201596+0000","last_fullsized":"2024-05-09T07:13:52.201596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698232+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:13:51.698232+0000","last_peered":"2024-05-09T07:13:51.698232+0000","last_clean":"2024-05-09T07:13:51.698232+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:13:51.698232+0000","last_undegraded":"2024-05-09T07:13:51.698232+0000","last_fullsized":"2024-05-09T07:13:51.698232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801528+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:13:51.801528+0000","last_peered":"2024-05-09T07:13:51.801528+0000","last_clean":"2024-05-09T07:13:51.801528+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:13:51.801528+0000","last_undegraded":"2024-05-09T07:13:51.801528+0000","last_fullsized":"2024-05-09T07:13:51.801528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491149+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:13:51.491149+0000","last_peered":"2024-05-09T07:13:51.491149+0000","last_clean":"2024-05-09T07:13:51.491149+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:13:51.491149+0000","last_undegraded":"2024-05-09T07:13:51.491149+0000","last_fullsized":"2024-05-09T07:13:51.491149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491381+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:13:51.491381+0000","last_peered":"2024-05-09T07:13:51.491381+0000","last_clean":"2024-05-09T07:13:51.491381+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:13:51.491381+0000","last_undegraded":"2024-05-09T07:13:51.491381+0000","last_fullsized":"2024-05-09T07:13:51.491381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801475+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:13:51.801475+0000","last_peered":"2024-05-09T07:13:51.801475+0000","last_clean":"2024-05-09T07:13:51.801475+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:13:51.801475+0000","last_undegraded":"2024-05-09T07:13:51.801475+0000","last_fullsized":"2024-05-09T07:13:51.801475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491169+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:13:51.491169+0000","last_peered":"2024-05-09T07:13:51.491169+0000","last_clean":"2024-05-09T07:13:51.491169+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:13:51.491169+0000","last_undegraded":"2024-05-09T07:13:51.491169+0000","last_fullsized":"2024-05-09T07:13:51.491169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490742+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:13:51.490742+0000","last_peered":"2024-05-09T07:13:51.490742+0000","last_clean":"2024-05-09T07:13:51.490742+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:13:51.490742+0000","last_undegraded":"2024-05-09T07:13:51.490742+0000","last_fullsized":"2024-05-09T07:13:51.490742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491961+0000","last_change":"2024-05-09T07:13:49.938952+0000","last_active":"2024-05-09T07:13:51.491961+0000","last_peered":"2024-05-09T07:13:51.491961+0000","last_clean":"2024-05-09T07:13:51.491961+0000","last_became_active":"2024-05-09T07:13:49.938552+0000","last_became_peered":"2024-05-09T07:13:49.938552+0000","last_unstale":"2024-05-09T07:13:51.491961+0000","last_undegraded":"2024-05-09T07:13:51.491961+0000","last_fullsized":"2024-05-09T07:13:51.491961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801398+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:13:51.801398+0000","last_peered":"2024-05-09T07:13:51.801398+0000","last_clean":"2024-05-09T07:13:51.801398+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:13:51.801398+0000","last_undegraded":"2024-05-09T07:13:51.801398+0000","last_fullsized":"2024-05-09T07:13:51.801398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697811+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:13:51.697811+0000","last_peered":"2024-05-09T07:13:51.697811+0000","last_clean":"2024-05-09T07:13:51.697811+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:13:51.697811+0000","last_undegraded":"2024-05-09T07:13:51.697811+0000","last_fullsized":"2024-05-09T07:13:51.697811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801577+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:13:51.801577+0000","last_peered":"2024-05-09T07:13:51.801577+0000","last_clean":"2024-05-09T07:13:51.801577+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:13:51.801577+0000","last_undegraded":"2024-05-09T07:13:51.801577+0000","last_fullsized":"2024-05-09T07:13:51.801577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199329+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:13:52.199329+0000","last_peered":"2024-05-09T07:13:52.199329+0000","last_clean":"2024-05-09T07:13:52.199329+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:13:52.199329+0000","last_undegraded":"2024-05-09T07:13:52.199329+0000","last_fullsized":"2024-05-09T07:13:52.199329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202341+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:13:52.202341+0000","last_peered":"2024-05-09T07:13:52.202341+0000","last_clean":"2024-05-09T07:13:52.202341+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:13:52.202341+0000","last_undegraded":"2024-05-09T07:13:52.202341+0000","last_fullsized":"2024-05-09T07:13:52.202341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800919+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:13:51.800919+0000","last_peered":"2024-05-09T07:13:51.800919+0000","last_clean":"2024-05-09T07:13:51.800919+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:13:51.800919+0000","last_undegraded":"2024-05-09T07:13:51.800919+0000","last_fullsized":"2024-05-09T07:13:51.800919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492207+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:13:51.492207+0000","last_peered":"2024-05-09T07:13:51.492207+0000","last_clean":"2024-05-09T07:13:51.492207+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:13:51.492207+0000","last_undegraded":"2024-05-09T07:13:51.492207+0000","last_fullsized":"2024-05-09T07:13:51.492207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491259+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:13:51.491259+0000","last_peered":"2024-05-09T07:13:51.491259+0000","last_clean":"2024-05-09T07:13:51.491259+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:13:51.491259+0000","last_undegraded":"2024-05-09T07:13:51.491259+0000","last_fullsized":"2024-05-09T07:13:51.491259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490763+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:13:51.490763+0000","last_peered":"2024-05-09T07:13:51.490763+0000","last_clean":"2024-05-09T07:13:51.490763+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:13:51.490763+0000","last_undegraded":"2024-05-09T07:13:51.490763+0000","last_fullsized":"2024-05-09T07:13:51.490763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491690+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:13:51.491690+0000","last_peered":"2024-05-09T07:13:51.491690+0000","last_clean":"2024-05-09T07:13:51.491690+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:13:51.491690+0000","last_undegraded":"2024-05-09T07:13:51.491690+0000","last_fullsized":"2024-05-09T07:13:51.491690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698566+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:13:51.698566+0000","last_peered":"2024-05-09T07:13:51.698566+0000","last_clean":"2024-05-09T07:13:51.698566+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:13:51.698566+0000","last_undegraded":"2024-05-09T07:13:51.698566+0000","last_fullsized":"2024-05-09T07:13:51.698566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198963+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:13:52.198963+0000","last_peered":"2024-05-09T07:13:52.198963+0000","last_clean":"2024-05-09T07:13:52.198963+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:13:52.198963+0000","last_undegraded":"2024-05-09T07:13:52.198963+0000","last_fullsized":"2024-05-09T07:13:52.198963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202806+0000","last_change":"2024-05-09T07:13:45.484793+0000","last_active":"2024-05-09T07:13:52.202806+0000","last_peered":"2024-05-09T07:13:52.202806+0000","last_clean":"2024-05-09T07:13:52.202806+0000","last_became_active":"2024-05-09T07:13:45.484248+0000","last_became_peered":"2024-05-09T07:13:45.484248+0000","last_unstale":"2024-05-09T07:13:52.202806+0000","last_undegraded":"2024-05-09T07:13:52.202806+0000","last_fullsized":"2024-05-09T07:13:52.202806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491769+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:13:51.491769+0000","last_peered":"2024-05-09T07:13:51.491769+0000","last_clean":"2024-05-09T07:13:51.491769+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:13:51.491769+0000","last_undegraded":"2024-05-09T07:13:51.491769+0000","last_fullsized":"2024-05-09T07:13:51.491769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491114+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:13:51.491114+0000","last_peered":"2024-05-09T07:13:51.491114+0000","last_clean":"2024-05-09T07:13:51.491114+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:13:51.491114+0000","last_undegraded":"2024-05-09T07:13:51.491114+0000","last_fullsized":"2024-05-09T07:13:51.491114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801532+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:13:51.801532+0000","last_peered":"2024-05-09T07:13:51.801532+0000","last_clean":"2024-05-09T07:13:51.801532+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:13:51.801532+0000","last_undegraded":"2024-05-09T07:13:51.801532+0000","last_fullsized":"2024-05-09T07:13:51.801532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202356+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:13:52.202356+0000","last_peered":"2024-05-09T07:13:52.202356+0000","last_clean":"2024-05-09T07:13:52.202356+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:13:52.202356+0000","last_undegraded":"2024-05-09T07:13:52.202356+0000","last_fullsized":"2024-05-09T07:13:52.202356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491646+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:13:51.491646+0000","last_peered":"2024-05-09T07:13:51.491646+0000","last_clean":"2024-05-09T07:13:51.491646+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:13:51.491646+0000","last_undegraded":"2024-05-09T07:13:51.491646+0000","last_fullsized":"2024-05-09T07:13:51.491646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490640+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:13:51.490640+0000","last_peered":"2024-05-09T07:13:51.490640+0000","last_clean":"2024-05-09T07:13:51.490640+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:13:51.490640+0000","last_undegraded":"2024-05-09T07:13:51.490640+0000","last_fullsized":"2024-05-09T07:13:51.490640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490075+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:13:51.490075+0000","last_peered":"2024-05-09T07:13:51.490075+0000","last_clean":"2024-05-09T07:13:51.490075+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:13:51.490075+0000","last_undegraded":"2024-05-09T07:13:51.490075+0000","last_fullsized":"2024-05-09T07:13:51.490075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492028+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:13:51.492028+0000","last_peered":"2024-05-09T07:13:51.492028+0000","last_clean":"2024-05-09T07:13:51.492028+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:13:51.492028+0000","last_undegraded":"2024-05-09T07:13:51.492028+0000","last_fullsized":"2024-05-09T07:13:51.492028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198795+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:13:52.198795+0000","last_peered":"2024-05-09T07:13:52.198795+0000","last_clean":"2024-05-09T07:13:52.198795+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:13:52.198795+0000","last_undegraded":"2024-05-09T07:13:52.198795+0000","last_fullsized":"2024-05-09T07:13:52.198795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697830+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:13:51.697830+0000","last_peered":"2024-05-09T07:13:51.697830+0000","last_clean":"2024-05-09T07:13:51.697830+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:13:51.697830+0000","last_undegraded":"2024-05-09T07:13:51.697830+0000","last_fullsized":"2024-05-09T07:13:51.697830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199305+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:13:52.199305+0000","last_peered":"2024-05-09T07:13:52.199305+0000","last_clean":"2024-05-09T07:13:52.199305+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:13:52.199305+0000","last_undegraded":"2024-05-09T07:13:52.199305+0000","last_fullsized":"2024-05-09T07:13:52.199305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491484+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:13:51.491484+0000","last_peered":"2024-05-09T07:13:51.491484+0000","last_clean":"2024-05-09T07:13:51.491484+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:13:51.491484+0000","last_undegraded":"2024-05-09T07:13:51.491484+0000","last_fullsized":"2024-05-09T07:13:51.491484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202232+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:13:52.202232+0000","last_peered":"2024-05-09T07:13:52.202232+0000","last_clean":"2024-05-09T07:13:52.202232+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:13:52.202232+0000","last_undegraded":"2024-05-09T07:13:52.202232+0000","last_fullsized":"2024-05-09T07:13:52.202232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697989+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:13:51.697989+0000","last_peered":"2024-05-09T07:13:51.697989+0000","last_clean":"2024-05-09T07:13:51.697989+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:13:51.697989+0000","last_undegraded":"2024-05-09T07:13:51.697989+0000","last_fullsized":"2024-05-09T07:13:51.697989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698344+0000","last_change":"2024-05-09T07:13:45.490251+0000","last_active":"2024-05-09T07:13:51.698344+0000","last_peered":"2024-05-09T07:13:51.698344+0000","last_clean":"2024-05-09T07:13:51.698344+0000","last_became_active":"2024-05-09T07:13:45.489707+0000","last_became_peered":"2024-05-09T07:13:45.489707+0000","last_unstale":"2024-05-09T07:13:51.698344+0000","last_undegraded":"2024-05-09T07:13:51.698344+0000","last_fullsized":"2024-05-09T07:13:51.698344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490413+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:13:51.490413+0000","last_peered":"2024-05-09T07:13:51.490413+0000","last_clean":"2024-05-09T07:13:51.490413+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:13:51.490413+0000","last_undegraded":"2024-05-09T07:13:51.490413+0000","last_fullsized":"2024-05-09T07:13:51.490413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201756+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:13:52.201756+0000","last_peered":"2024-05-09T07:13:52.201756+0000","last_clean":"2024-05-09T07:13:52.201756+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:13:52.201756+0000","last_undegraded":"2024-05-09T07:13:52.201756+0000","last_fullsized":"2024-05-09T07:13:52.201756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202663+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:13:52.202663+0000","last_peered":"2024-05-09T07:13:52.202663+0000","last_clean":"2024-05-09T07:13:52.202663+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:13:52.202663+0000","last_undegraded":"2024-05-09T07:13:52.202663+0000","last_fullsized":"2024-05-09T07:13:52.202663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490395+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:13:51.490395+0000","last_peered":"2024-05-09T07:13:51.490395+0000","last_clean":"2024-05-09T07:13:51.490395+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:13:51.490395+0000","last_undegraded":"2024-05-09T07:13:51.490395+0000","last_fullsized":"2024-05-09T07:13:51.490395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199283+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:13:52.199283+0000","last_peered":"2024-05-09T07:13:52.199283+0000","last_clean":"2024-05-09T07:13:52.199283+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:13:52.199283+0000","last_undegraded":"2024-05-09T07:13:52.199283+0000","last_fullsized":"2024-05-09T07:13:52.199283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698652+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:13:51.698652+0000","last_peered":"2024-05-09T07:13:51.698652+0000","last_clean":"2024-05-09T07:13:51.698652+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:13:51.698652+0000","last_undegraded":"2024-05-09T07:13:51.698652+0000","last_fullsized":"2024-05-09T07:13:51.698652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490223+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:13:51.490223+0000","last_peered":"2024-05-09T07:13:51.490223+0000","last_clean":"2024-05-09T07:13:51.490223+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:13:51.490223+0000","last_undegraded":"2024-05-09T07:13:51.490223+0000","last_fullsized":"2024-05-09T07:13:51.490223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201384+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:13:52.201384+0000","last_peered":"2024-05-09T07:13:52.201384+0000","last_clean":"2024-05-09T07:13:52.201384+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:13:52.201384+0000","last_undegraded":"2024-05-09T07:13:52.201384+0000","last_fullsized":"2024-05-09T07:13:52.201384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698489+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:13:51.698489+0000","last_peered":"2024-05-09T07:13:51.698489+0000","last_clean":"2024-05-09T07:13:51.698489+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:13:51.698489+0000","last_undegraded":"2024-05-09T07:13:51.698489+0000","last_fullsized":"2024-05-09T07:13:51.698489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801555+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:13:51.801555+0000","last_peered":"2024-05-09T07:13:51.801555+0000","last_clean":"2024-05-09T07:13:51.801555+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:13:51.801555+0000","last_undegraded":"2024-05-09T07:13:51.801555+0000","last_fullsized":"2024-05-09T07:13:51.801555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800811+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:13:51.800811+0000","last_peered":"2024-05-09T07:13:51.800811+0000","last_clean":"2024-05-09T07:13:51.800811+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:13:51.800811+0000","last_undegraded":"2024-05-09T07:13:51.800811+0000","last_fullsized":"2024-05-09T07:13:51.800811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698054+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:13:51.698054+0000","last_peered":"2024-05-09T07:13:51.698054+0000","last_clean":"2024-05-09T07:13:51.698054+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:13:51.698054+0000","last_undegraded":"2024-05-09T07:13:51.698054+0000","last_fullsized":"2024-05-09T07:13:51.698054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489838+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:13:51.489838+0000","last_peered":"2024-05-09T07:13:51.489838+0000","last_clean":"2024-05-09T07:13:51.489838+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:13:51.489838+0000","last_undegraded":"2024-05-09T07:13:51.489838+0000","last_fullsized":"2024-05-09T07:13:51.489838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199184+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:13:52.199184+0000","last_peered":"2024-05-09T07:13:52.199184+0000","last_clean":"2024-05-09T07:13:52.199184+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:13:52.199184+0000","last_undegraded":"2024-05-09T07:13:52.199184+0000","last_fullsized":"2024-05-09T07:13:52.199184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201678+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:13:52.201678+0000","last_peered":"2024-05-09T07:13:52.201678+0000","last_clean":"2024-05-09T07:13:52.201678+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:13:52.201678+0000","last_undegraded":"2024-05-09T07:13:52.201678+0000","last_fullsized":"2024-05-09T07:13:52.201678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201895+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:13:52.201895+0000","last_peered":"2024-05-09T07:13:52.201895+0000","last_clean":"2024-05-09T07:13:52.201895+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:13:52.201895+0000","last_undegraded":"2024-05-09T07:13:52.201895+0000","last_fullsized":"2024-05-09T07:13:52.201895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492100+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:13:51.492100+0000","last_peered":"2024-05-09T07:13:51.492100+0000","last_clean":"2024-05-09T07:13:51.492100+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:13:51.492100+0000","last_undegraded":"2024-05-09T07:13:51.492100+0000","last_fullsized":"2024-05-09T07:13:51.492100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698135+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:13:51.698135+0000","last_peered":"2024-05-09T07:13:51.698135+0000","last_clean":"2024-05-09T07:13:51.698135+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:13:51.698135+0000","last_undegraded":"2024-05-09T07:13:51.698135+0000","last_fullsized":"2024-05-09T07:13:51.698135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202333+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:13:52.202333+0000","last_peered":"2024-05-09T07:13:52.202333+0000","last_clean":"2024-05-09T07:13:52.202333+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:13:52.202333+0000","last_undegraded":"2024-05-09T07:13:52.202333+0000","last_fullsized":"2024-05-09T07:13:52.202333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491909+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:13:51.491909+0000","last_peered":"2024-05-09T07:13:51.491909+0000","last_clean":"2024-05-09T07:13:51.491909+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:13:51.491909+0000","last_undegraded":"2024-05-09T07:13:51.491909+0000","last_fullsized":"2024-05-09T07:13:51.491909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198932+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:13:52.198932+0000","last_peered":"2024-05-09T07:13:52.198932+0000","last_clean":"2024-05-09T07:13:52.198932+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:13:52.198932+0000","last_undegraded":"2024-05-09T07:13:52.198932+0000","last_fullsized":"2024-05-09T07:13:52.198932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199299+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:13:52.199299+0000","last_peered":"2024-05-09T07:13:52.199299+0000","last_clean":"2024-05-09T07:13:52.199299+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:13:52.199299+0000","last_undegraded":"2024-05-09T07:13:52.199299+0000","last_fullsized":"2024-05-09T07:13:52.199299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202849+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:13:52.202849+0000","last_peered":"2024-05-09T07:13:52.202849+0000","last_clean":"2024-05-09T07:13:52.202849+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:13:52.202849+0000","last_undegraded":"2024-05-09T07:13:52.202849+0000","last_fullsized":"2024-05-09T07:13:52.202849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698233+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:13:51.698233+0000","last_peered":"2024-05-09T07:13:51.698233+0000","last_clean":"2024-05-09T07:13:51.698233+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:13:51.698233+0000","last_undegraded":"2024-05-09T07:13:51.698233+0000","last_fullsized":"2024-05-09T07:13:51.698233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:54.939707+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:13:54.939707+0000","last_peered":"2024-05-09T07:13:54.939707+0000","last_clean":"2024-05-09T07:13:54.939707+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:13:54.939707+0000","last_undegraded":"2024-05-09T07:13:54.939707+0000","last_fullsized":"2024-05-09T07:13:54.939707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492117+0000","last_change":"2024-05-09T07:13:49.503824+0000","last_active":"2024-05-09T07:13:51.492117+0000","last_peered":"2024-05-09T07:13:51.492117+0000","last_clean":"2024-05-09T07:13:51.492117+0000","last_became_active":"2024-05-09T07:13:49.503453+0000","last_became_peered":"2024-05-09T07:13:49.503453+0000","last_unstale":"2024-05-09T07:13:51.492117+0000","last_undegraded":"2024-05-09T07:13:51.492117+0000","last_fullsized":"2024-05-09T07:13:51.492117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801449+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:13:51.801449+0000","last_peered":"2024-05-09T07:13:51.801449+0000","last_clean":"2024-05-09T07:13:51.801449+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:13:51.801449+0000","last_undegraded":"2024-05-09T07:13:51.801449+0000","last_fullsized":"2024-05-09T07:13:51.801449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490306+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:13:51.490306+0000","last_peered":"2024-05-09T07:13:51.490306+0000","last_clean":"2024-05-09T07:13:51.490306+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:13:51.490306+0000","last_undegraded":"2024-05-09T07:13:51.490306+0000","last_fullsized":"2024-05-09T07:13:51.490306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801327+0000","last_change":"2024-05-09T07:13:49.509136+0000","last_active":"2024-05-09T07:13:51.801327+0000","last_peered":"2024-05-09T07:13:51.801327+0000","last_clean":"2024-05-09T07:13:51.801327+0000","last_became_active":"2024-05-09T07:13:49.508102+0000","last_became_peered":"2024-05-09T07:13:49.508102+0000","last_unstale":"2024-05-09T07:13:51.801327+0000","last_undegraded":"2024-05-09T07:13:51.801327+0000","last_fullsized":"2024-05-09T07:13:51.801327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199289+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:13:52.199289+0000","last_peered":"2024-05-09T07:13:52.199289+0000","last_clean":"2024-05-09T07:13:52.199289+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:13:52.199289+0000","last_undegraded":"2024-05-09T07:13:52.199289+0000","last_fullsized":"2024-05-09T07:13:52.199289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202839+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:13:52.202839+0000","last_peered":"2024-05-09T07:13:52.202839+0000","last_clean":"2024-05-09T07:13:52.202839+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:13:52.202839+0000","last_undegraded":"2024-05-09T07:13:52.202839+0000","last_fullsized":"2024-05-09T07:13:52.202839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492108+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:13:51.492108+0000","last_peered":"2024-05-09T07:13:51.492108+0000","last_clean":"2024-05-09T07:13:51.492108+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:13:51.492108+0000","last_undegraded":"2024-05-09T07:13:51.492108+0000","last_fullsized":"2024-05-09T07:13:51.492108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202701+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:13:52.202701+0000","last_peered":"2024-05-09T07:13:52.202701+0000","last_clean":"2024-05-09T07:13:52.202701+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:13:52.202701+0000","last_undegraded":"2024-05-09T07:13:52.202701+0000","last_fullsized":"2024-05-09T07:13:52.202701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202142+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:13:52.202142+0000","last_peered":"2024-05-09T07:13:52.202142+0000","last_clean":"2024-05-09T07:13:52.202142+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:13:52.202142+0000","last_undegraded":"2024-05-09T07:13:52.202142+0000","last_fullsized":"2024-05-09T07:13:52.202142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491246+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:13:51.491246+0000","last_peered":"2024-05-09T07:13:51.491246+0000","last_clean":"2024-05-09T07:13:51.491246+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:13:51.491246+0000","last_undegraded":"2024-05-09T07:13:51.491246+0000","last_fullsized":"2024-05-09T07:13:51.491246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199132+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:13:52.199132+0000","last_peered":"2024-05-09T07:13:52.199132+0000","last_clean":"2024-05-09T07:13:52.199132+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:13:52.199132+0000","last_undegraded":"2024-05-09T07:13:52.199132+0000","last_fullsized":"2024-05-09T07:13:52.199132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490579+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:13:51.490579+0000","last_peered":"2024-05-09T07:13:51.490579+0000","last_clean":"2024-05-09T07:13:51.490579+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:13:51.490579+0000","last_undegraded":"2024-05-09T07:13:51.490579+0000","last_fullsized":"2024-05-09T07:13:51.490579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201450+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:13:52.201450+0000","last_peered":"2024-05-09T07:13:52.201450+0000","last_clean":"2024-05-09T07:13:52.201450+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:13:52.201450+0000","last_undegraded":"2024-05-09T07:13:52.201450+0000","last_fullsized":"2024-05-09T07:13:52.201450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202518+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:13:52.202518+0000","last_peered":"2024-05-09T07:13:52.202518+0000","last_clean":"2024-05-09T07:13:52.202518+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:13:52.202518+0000","last_undegraded":"2024-05-09T07:13:52.202518+0000","last_fullsized":"2024-05-09T07:13:52.202518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491038+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:13:51.491038+0000","last_peered":"2024-05-09T07:13:51.491038+0000","last_clean":"2024-05-09T07:13:51.491038+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:13:51.491038+0000","last_undegraded":"2024-05-09T07:13:51.491038+0000","last_fullsized":"2024-05-09T07:13:51.491038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491161+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:13:51.491161+0000","last_peered":"2024-05-09T07:13:51.491161+0000","last_clean":"2024-05-09T07:13:51.491161+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:13:51.491161+0000","last_undegraded":"2024-05-09T07:13:51.491161+0000","last_fullsized":"2024-05-09T07:13:51.491161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201962+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:13:52.201962+0000","last_peered":"2024-05-09T07:13:52.201962+0000","last_clean":"2024-05-09T07:13:52.201962+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:13:52.201962+0000","last_undegraded":"2024-05-09T07:13:52.201962+0000","last_fullsized":"2024-05-09T07:13:52.201962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199171+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:13:52.199171+0000","last_peered":"2024-05-09T07:13:52.199171+0000","last_clean":"2024-05-09T07:13:52.199171+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:13:52.199171+0000","last_undegraded":"2024-05-09T07:13:52.199171+0000","last_fullsized":"2024-05-09T07:13:52.199171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490242+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:13:51.490242+0000","last_peered":"2024-05-09T07:13:51.490242+0000","last_clean":"2024-05-09T07:13:51.490242+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:13:51.490242+0000","last_undegraded":"2024-05-09T07:13:51.490242+0000","last_fullsized":"2024-05-09T07:13:51.490242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491319+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:13:51.491319+0000","last_peered":"2024-05-09T07:13:51.491319+0000","last_clean":"2024-05-09T07:13:51.491319+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:13:51.491319+0000","last_undegraded":"2024-05-09T07:13:51.491319+0000","last_fullsized":"2024-05-09T07:13:51.491319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801024+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:13:51.801024+0000","last_peered":"2024-05-09T07:13:51.801024+0000","last_clean":"2024-05-09T07:13:51.801024+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:13:51.801024+0000","last_undegraded":"2024-05-09T07:13:51.801024+0000","last_fullsized":"2024-05-09T07:13:51.801024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492243+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:13:51.492243+0000","last_peered":"2024-05-09T07:13:51.492243+0000","last_clean":"2024-05-09T07:13:51.492243+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:13:51.492243+0000","last_undegraded":"2024-05-09T07:13:51.492243+0000","last_fullsized":"2024-05-09T07:13:51.492243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201566+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:13:52.201566+0000","last_peered":"2024-05-09T07:13:52.201566+0000","last_clean":"2024-05-09T07:13:52.201566+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:13:52.201566+0000","last_undegraded":"2024-05-09T07:13:52.201566+0000","last_fullsized":"2024-05-09T07:13:52.201566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489979+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:13:51.489979+0000","last_peered":"2024-05-09T07:13:51.489979+0000","last_clean":"2024-05-09T07:13:51.489979+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:13:51.489979+0000","last_undegraded":"2024-05-09T07:13:51.489979+0000","last_fullsized":"2024-05-09T07:13:51.489979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202280+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:13:52.202280+0000","last_peered":"2024-05-09T07:13:52.202280+0000","last_clean":"2024-05-09T07:13:52.202280+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:13:52.202280+0000","last_undegraded":"2024-05-09T07:13:52.202280+0000","last_fullsized":"2024-05-09T07:13:52.202280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199032+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:13:52.199032+0000","last_peered":"2024-05-09T07:13:52.199032+0000","last_clean":"2024-05-09T07:13:52.199032+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:13:52.199032+0000","last_undegraded":"2024-05-09T07:13:52.199032+0000","last_fullsized":"2024-05-09T07:13:52.199032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491975+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:13:51.491975+0000","last_peered":"2024-05-09T07:13:51.491975+0000","last_clean":"2024-05-09T07:13:51.491975+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:13:51.491975+0000","last_undegraded":"2024-05-09T07:13:51.491975+0000","last_fullsized":"2024-05-09T07:13:51.491975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490135+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:13:51.490135+0000","last_peered":"2024-05-09T07:13:51.490135+0000","last_clean":"2024-05-09T07:13:51.490135+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:13:51.490135+0000","last_undegraded":"2024-05-09T07:13:51.490135+0000","last_fullsized":"2024-05-09T07:13:51.490135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202331+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:13:52.202331+0000","last_peered":"2024-05-09T07:13:52.202331+0000","last_clean":"2024-05-09T07:13:52.202331+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:13:52.202331+0000","last_undegraded":"2024-05-09T07:13:52.202331+0000","last_fullsized":"2024-05-09T07:13:52.202331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492151+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:13:51.492151+0000","last_peered":"2024-05-09T07:13:51.492151+0000","last_clean":"2024-05-09T07:13:51.492151+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:13:51.492151+0000","last_undegraded":"2024-05-09T07:13:51.492151+0000","last_fullsized":"2024-05-09T07:13:51.492151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201443+0000","last_change":"2024-05-09T07:13:45.483771+0000","last_active":"2024-05-09T07:13:52.201443+0000","last_peered":"2024-05-09T07:13:52.201443+0000","last_clean":"2024-05-09T07:13:52.201443+0000","last_became_active":"2024-05-09T07:13:45.483484+0000","last_became_peered":"2024-05-09T07:13:45.483484+0000","last_unstale":"2024-05-09T07:13:52.201443+0000","last_undegraded":"2024-05-09T07:13:52.201443+0000","last_fullsized":"2024-05-09T07:13:52.201443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491076+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:13:51.491076+0000","last_peered":"2024-05-09T07:13:51.491076+0000","last_clean":"2024-05-09T07:13:51.491076+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:13:51.491076+0000","last_undegraded":"2024-05-09T07:13:51.491076+0000","last_fullsized":"2024-05-09T07:13:51.491076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202200+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:13:52.202200+0000","last_peered":"2024-05-09T07:13:52.202200+0000","last_clean":"2024-05-09T07:13:52.202200+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:13:52.202200+0000","last_undegraded":"2024-05-09T07:13:52.202200+0000","last_fullsized":"2024-05-09T07:13:52.202200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198920+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:13:52.198920+0000","last_peered":"2024-05-09T07:13:52.198920+0000","last_clean":"2024-05-09T07:13:52.198920+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:13:52.198920+0000","last_undegraded":"2024-05-09T07:13:52.198920+0000","last_fullsized":"2024-05-09T07:13:52.198920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698479+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:13:51.698479+0000","last_peered":"2024-05-09T07:13:51.698479+0000","last_clean":"2024-05-09T07:13:51.698479+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:13:51.698479+0000","last_undegraded":"2024-05-09T07:13:51.698479+0000","last_fullsized":"2024-05-09T07:13:51.698479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490136+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:13:51.490136+0000","last_peered":"2024-05-09T07:13:51.490136+0000","last_clean":"2024-05-09T07:13:51.490136+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:13:51.490136+0000","last_undegraded":"2024-05-09T07:13:51.490136+0000","last_fullsized":"2024-05-09T07:13:51.490136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202983+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:13:52.202983+0000","last_peered":"2024-05-09T07:13:52.202983+0000","last_clean":"2024-05-09T07:13:52.202983+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:13:52.202983+0000","last_undegraded":"2024-05-09T07:13:52.202983+0000","last_fullsized":"2024-05-09T07:13:52.202983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201343+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:13:52.201343+0000","last_peered":"2024-05-09T07:13:52.201343+0000","last_clean":"2024-05-09T07:13:52.201343+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:13:52.201343+0000","last_undegraded":"2024-05-09T07:13:52.201343+0000","last_fullsized":"2024-05-09T07:13:52.201343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490661+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:13:51.490661+0000","last_peered":"2024-05-09T07:13:51.490661+0000","last_clean":"2024-05-09T07:13:51.490661+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:13:51.490661+0000","last_undegraded":"2024-05-09T07:13:51.490661+0000","last_fullsized":"2024-05-09T07:13:51.490661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198914+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:13:52.198914+0000","last_peered":"2024-05-09T07:13:52.198914+0000","last_clean":"2024-05-09T07:13:52.198914+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:13:52.198914+0000","last_undegraded":"2024-05-09T07:13:52.198914+0000","last_fullsized":"2024-05-09T07:13:52.198914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491230+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:13:51.491230+0000","last_peered":"2024-05-09T07:13:51.491230+0000","last_clean":"2024-05-09T07:13:51.491230+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:13:51.491230+0000","last_undegraded":"2024-05-09T07:13:51.491230+0000","last_fullsized":"2024-05-09T07:13:51.491230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801266+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:13:51.801266+0000","last_peered":"2024-05-09T07:13:51.801266+0000","last_clean":"2024-05-09T07:13:51.801266+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:13:51.801266+0000","last_undegraded":"2024-05-09T07:13:51.801266+0000","last_fullsized":"2024-05-09T07:13:51.801266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202141+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:13:52.202141+0000","last_peered":"2024-05-09T07:13:52.202141+0000","last_clean":"2024-05-09T07:13:52.202141+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:13:52.202141+0000","last_undegraded":"2024-05-09T07:13:52.202141+0000","last_fullsized":"2024-05-09T07:13:52.202141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202227+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:13:52.202227+0000","last_peered":"2024-05-09T07:13:52.202227+0000","last_clean":"2024-05-09T07:13:52.202227+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:13:52.202227+0000","last_undegraded":"2024-05-09T07:13:52.202227+0000","last_fullsized":"2024-05-09T07:13:52.202227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698631+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:13:51.698631+0000","last_peered":"2024-05-09T07:13:51.698631+0000","last_clean":"2024-05-09T07:13:51.698631+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:13:51.698631+0000","last_undegraded":"2024-05-09T07:13:51.698631+0000","last_fullsized":"2024-05-09T07:13:51.698631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490881+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:13:51.490881+0000","last_peered":"2024-05-09T07:13:51.490881+0000","last_clean":"2024-05-09T07:13:51.490881+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:13:51.490881+0000","last_undegraded":"2024-05-09T07:13:51.490881+0000","last_fullsized":"2024-05-09T07:13:51.490881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698046+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:13:51.698046+0000","last_peered":"2024-05-09T07:13:51.698046+0000","last_clean":"2024-05-09T07:13:51.698046+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:13:51.698046+0000","last_undegraded":"2024-05-09T07:13:51.698046+0000","last_fullsized":"2024-05-09T07:13:51.698046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202156+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:13:52.202156+0000","last_peered":"2024-05-09T07:13:52.202156+0000","last_clean":"2024-05-09T07:13:52.202156+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:13:52.202156+0000","last_undegraded":"2024-05-09T07:13:52.202156+0000","last_fullsized":"2024-05-09T07:13:52.202156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698092+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:13:51.698092+0000","last_peered":"2024-05-09T07:13:51.698092+0000","last_clean":"2024-05-09T07:13:51.698092+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:13:51.698092+0000","last_undegraded":"2024-05-09T07:13:51.698092+0000","last_fullsized":"2024-05-09T07:13:51.698092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202561+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:13:52.202561+0000","last_peered":"2024-05-09T07:13:52.202561+0000","last_clean":"2024-05-09T07:13:52.202561+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:13:52.202561+0000","last_undegraded":"2024-05-09T07:13:52.202561+0000","last_fullsized":"2024-05-09T07:13:52.202561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199052+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:13:52.199052+0000","last_peered":"2024-05-09T07:13:52.199052+0000","last_clean":"2024-05-09T07:13:52.199052+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:13:52.199052+0000","last_undegraded":"2024-05-09T07:13:52.199052+0000","last_fullsized":"2024-05-09T07:13:52.199052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491163+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:13:51.491163+0000","last_peered":"2024-05-09T07:13:51.491163+0000","last_clean":"2024-05-09T07:13:51.491163+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:13:51.491163+0000","last_undegraded":"2024-05-09T07:13:51.491163+0000","last_fullsized":"2024-05-09T07:13:51.491163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698522+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:13:51.698522+0000","last_peered":"2024-05-09T07:13:51.698522+0000","last_clean":"2024-05-09T07:13:51.698522+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:13:51.698522+0000","last_undegraded":"2024-05-09T07:13:51.698522+0000","last_fullsized":"2024-05-09T07:13:51.698522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:14:06.887 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:14:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.021+0000 7f61dec9a640 1 Processor -- start 2024-05-09T07:14:07.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.021+0000 7f61dec9a640 1 -- start start 2024-05-09T07:14:07.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dec9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d806c670 0x7f61d806ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dec9a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f61d80c6460 0x7f61d80c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dec9a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61d80c19f0 0x7f61d80c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dec9a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f61d80bdb40 con 0x7f61d80c19f0 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dec9a640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f61d81523f0 con 0x7f61d80c6460 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dec9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61d80c47d0 con 0x7f61d806c670 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61d80c19f0 0x7f61d80c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f61d80c19f0 0x7f61d80c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44284/0 (socket says 172.21.15.160:44284) 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 -- 172.21.15.160:0/2271011289 learned_addr learned my addr 172.21.15.160:0/2271011289 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61d7fff640 1 --2- 172.21.15.160:0/2271011289 >> v2:172.21.15.188:3300/0 conn(0x7f61d80c6460 0x7f61d80c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dca36640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d806c670 0x7f61d806ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 -- 172.21.15.160:0/2271011289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d806c670 msgr2=0x7f61d806ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d806c670 0x7f61d806ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 -- 172.21.15.160:0/2271011289 >> v2:172.21.15.188:3300/0 conn(0x7f61d80c6460 msgr2=0x7f61d80c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 --2- 172.21.15.160:0/2271011289 >> v2:172.21.15.188:3300/0 conn(0x7f61d80c6460 0x7f61d80c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.022+0000 7f61dd237640 1 -- 172.21.15.160:0/2271011289 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d8152920 con 0x7f61d80c19f0 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.023+0000 7f61dca36640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d806c670 0x7f61d806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.023+0000 7f61dd237640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d80c19f0 0x7f61d80c2c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61d80a9740 tx=0x7f61d002c780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491fc279e3b3f6df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.023+0000 7f61d77fe640 1 -- 172.21.15.160:0/2271011289 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0009600 con 0x7f61d80c19f0 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.023+0000 7f61d77fe640 1 -- 172.21.15.160:0/2271011289 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61d00097a0 con 0x7f61d80c19f0 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.023+0000 7f61d77fe640 1 -- 172.21.15.160:0/2271011289 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d0009b70 con 0x7f61d80c19f0 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 -- 172.21.15.160:0/2271011289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d80c19f0 msgr2=0x7f61d80c2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d80c19f0 0x7f61d80c2c60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f61d80a9740 tx=0x7f61d002c780 comp rx=0 tx=0).stop 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 -- 172.21.15.160:0/2271011289 shutdown_connections 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d80c19f0 0x7f61d80c2c60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2271011289 >> v2:172.21.15.188:3300/0 conn(0x7f61d80c6460 0x7f61d80c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.027 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2271011289 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d806c670 0x7f61d806ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 -- 172.21.15.160:0/2271011289 >> 172.21.15.160:0/2271011289 conn(0x7f61d80c7580 msgr2=0x7f61d80a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:07.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 -- 172.21.15.160:0/2271011289 shutdown_connections 2024-05-09T07:14:07.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.024+0000 7f61dec9a640 1 -- 172.21.15.160:0/2271011289 wait complete. 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.025+0000 7f61dec9a640 1 Processor -- start 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.025+0000 7f61dec9a640 1 -- start start 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dec9a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 0x7f61d81dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dec9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d80c19f0 0x7f61d81eb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dec9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61d80c6460 0x7f61d814c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dec9a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f61d8152640 con 0x7f61d806c670 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dec9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f61d80a6170 con 0x7f61d80c6460 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dec9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f61d80c2200 con 0x7f61d80c19f0 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 0x7f61d81dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 0x7f61d81dc3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44294/0 (socket says 172.21.15.160:44294) 2024-05-09T07:14:07.030 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 -- 172.21.15.160:0/2480189450 learned_addr learned my addr 172.21.15.160:0/2480189450 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61d7fff640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d80c19f0 0x7f61d81eb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 -- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d80c19f0 msgr2=0x7f61d81eb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dd237640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61d80c6460 0x7f61d814c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d80c19f0 0x7f61d81eb460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 -- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61d80c6460 msgr2=0x7f61d814c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61d80c6460 0x7f61d814c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 -- 172.21.15.160:0/2480189450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d80aab30 con 0x7f61d806c670 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.026+0000 7f61dca36640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 0x7f61d81dc3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f61c800c7c0 tx=0x7f61c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.027+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8015070 con 0x7f61d806c670 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.027+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61c80079d0 con 0x7f61d806c670 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.027+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d81ef140 con 0x7f61d806c670 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.027+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8007da0 con 0x7f61d806c670 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.027+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d80aba60 con 0x7f61d806c670 2024-05-09T07:14:07.031 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.028+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61d80756c0 con 0x7f61d806c670 2024-05-09T07:14:07.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.028+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f61c8018070 con 0x7f61d806c670 2024-05-09T07:14:07.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.029+0000 7f61d57fa640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61c406be30 0x7f61c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.029+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f61c808afb0 con 0x7f61d806c670 2024-05-09T07:14:07.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.029+0000 7f61d7fff640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61c406be30 0x7f61c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.032 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.029+0000 7f61d7fff640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61c406be30 0x7f61c406e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006fd0 tx=0x7f61cc008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.034 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.033+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f61c80596b0 con 0x7f61d806c670 2024-05-09T07:14:07.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.205+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61d806beb0 con 0x7f61c406be30 2024-05-09T07:14:07.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.212+0000 7f61d57fa640 1 -- 172.21.15.160:0/2480189450 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f61d806beb0 con 0x7f61c406be30 2024-05-09T07:14:07.218 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:07.220 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61c406be30 msgr2=0x7f61c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61c406be30 0x7f61c406e2f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f61cc006fd0 tx=0x7f61cc008040 comp rx=0 tx=0).stop 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 msgr2=0x7f61d81dc3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 0x7f61d81dc3c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f61c800c7c0 tx=0x7f61c800cc80 comp rx=0 tx=0).stop 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 shutdown_connections 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f61c406be30 0x7f61c406e2f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.222 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f61d80c6460 0x7f61d814c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f61d80c19f0 0x7f61d81eb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 --2- 172.21.15.160:0/2480189450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61d806c670 0x7f61d81dc3c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 >> 172.21.15.160:0/2480189450 conn(0x7f61d80c7580 msgr2=0x7f61d814edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:07.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 shutdown_connections 2024-05-09T07:14:07.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.216+0000 7f61dec9a640 1 -- 172.21.15.160:0/2480189450 wait complete. 2024-05-09T07:14:07.229 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2024-05-09T07:14:07.183213+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219464,"kb_used_data":3304,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755256,"statfs":{"total":773094113280,"available":772869382144,"internally_reserved":0,"allocated":3383296,"data_stored":2049160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002521"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199324+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:13:52.199324+0000","last_peered":"2024-05-09T07:13:52.199324+0000","last_clean":"2024-05-09T07:13:52.199324+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:13:52.199324+0000","last_undegraded":"2024-05-09T07:13:52.199324+0000","last_fullsized":"2024-05-09T07:13:52.199324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491364+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:13:51.491364+0000","last_peered":"2024-05-09T07:13:51.491364+0000","last_clean":"2024-05-09T07:13:51.491364+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:13:51.491364+0000","last_undegraded":"2024-05-09T07:13:51.491364+0000","last_fullsized":"2024-05-09T07:13:51.491364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490779+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:13:51.490779+0000","last_peered":"2024-05-09T07:13:51.490779+0000","last_clean":"2024-05-09T07:13:51.490779+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:13:51.490779+0000","last_undegraded":"2024-05-09T07:13:51.490779+0000","last_fullsized":"2024-05-09T07:13:51.490779+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491562+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:13:51.491562+0000","last_peered":"2024-05-09T07:13:51.491562+0000","last_clean":"2024-05-09T07:13:51.491562+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:13:51.491562+0000","last_undegraded":"2024-05-09T07:13:51.491562+0000","last_fullsized":"2024-05-09T07:13:51.491562+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199187+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:13:52.199187+0000","last_peered":"2024-05-09T07:13:52.199187+0000","last_clean":"2024-05-09T07:13:52.199187+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:13:52.199187+0000","last_undegraded":"2024-05-09T07:13:52.199187+0000","last_fullsized":"2024-05-09T07:13:52.199187+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800899+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:13:51.800899+0000","last_peered":"2024-05-09T07:13:51.800899+0000","last_clean":"2024-05-09T07:13:51.800899+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:13:51.800899+0000","last_undegraded":"2024-05-09T07:13:51.800899+0000","last_fullsized":"2024-05-09T07:13:51.800899+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698480+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:13:51.698480+0000","last_peered":"2024-05-09T07:13:51.698480+0000","last_clean":"2024-05-09T07:13:51.698480+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:13:51.698480+0000","last_undegraded":"2024-05-09T07:13:51.698480+0000","last_fullsized":"2024-05-09T07:13:51.698480+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202587+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:13:52.202587+0000","last_peered":"2024-05-09T07:13:52.202587+0000","last_clean":"2024-05-09T07:13:52.202587+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:13:52.202587+0000","last_undegraded":"2024-05-09T07:13:52.202587+0000","last_fullsized":"2024-05-09T07:13:52.202587+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801107+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:13:51.801107+0000","last_peered":"2024-05-09T07:13:51.801107+0000","last_clean":"2024-05-09T07:13:51.801107+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:13:51.801107+0000","last_undegraded":"2024-05-09T07:13:51.801107+0000","last_fullsized":"2024-05-09T07:13:51.801107+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698221+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:13:51.698221+0000","last_peered":"2024-05-09T07:13:51.698221+0000","last_clean":"2024-05-09T07:13:51.698221+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:13:51.698221+0000","last_undegraded":"2024-05-09T07:13:51.698221+0000","last_fullsized":"2024-05-09T07:13:51.698221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202703+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:13:52.202703+0000","last_peered":"2024-05-09T07:13:52.202703+0000","last_clean":"2024-05-09T07:13:52.202703+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:13:52.202703+0000","last_undegraded":"2024-05-09T07:13:52.202703+0000","last_fullsized":"2024-05-09T07:13:52.202703+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800950+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:13:51.800950+0000","last_peered":"2024-05-09T07:13:51.800950+0000","last_clean":"2024-05-09T07:13:51.800950+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:13:51.800950+0000","last_undegraded":"2024-05-09T07:13:51.800950+0000","last_fullsized":"2024-05-09T07:13:51.800950+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201596+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:13:52.201596+0000","last_peered":"2024-05-09T07:13:52.201596+0000","last_clean":"2024-05-09T07:13:52.201596+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:13:52.201596+0000","last_undegraded":"2024-05-09T07:13:52.201596+0000","last_fullsized":"2024-05-09T07:13:52.201596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698232+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:13:51.698232+0000","last_peered":"2024-05-09T07:13:51.698232+0000","last_clean":"2024-05-09T07:13:51.698232+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:13:51.698232+0000","last_undegraded":"2024-05-09T07:13:51.698232+0000","last_fullsized":"2024-05-09T07:13:51.698232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801528+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:13:51.801528+0000","last_peered":"2024-05-09T07:13:51.801528+0000","last_clean":"2024-05-09T07:13:51.801528+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:13:51.801528+0000","last_undegraded":"2024-05-09T07:13:51.801528+0000","last_fullsized":"2024-05-09T07:13:51.801528+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491149+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:13:51.491149+0000","last_peered":"2024-05-09T07:13:51.491149+0000","last_clean":"2024-05-09T07:13:51.491149+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:13:51.491149+0000","last_undegraded":"2024-05-09T07:13:51.491149+0000","last_fullsized":"2024-05-09T07:13:51.491149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491381+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:13:51.491381+0000","last_peered":"2024-05-09T07:13:51.491381+0000","last_clean":"2024-05-09T07:13:51.491381+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:13:51.491381+0000","last_undegraded":"2024-05-09T07:13:51.491381+0000","last_fullsized":"2024-05-09T07:13:51.491381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801475+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:13:51.801475+0000","last_peered":"2024-05-09T07:13:51.801475+0000","last_clean":"2024-05-09T07:13:51.801475+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:13:51.801475+0000","last_undegraded":"2024-05-09T07:13:51.801475+0000","last_fullsized":"2024-05-09T07:13:51.801475+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491169+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:13:51.491169+0000","last_peered":"2024-05-09T07:13:51.491169+0000","last_clean":"2024-05-09T07:13:51.491169+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:13:51.491169+0000","last_undegraded":"2024-05-09T07:13:51.491169+0000","last_fullsized":"2024-05-09T07:13:51.491169+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490742+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:13:51.490742+0000","last_peered":"2024-05-09T07:13:51.490742+0000","last_clean":"2024-05-09T07:13:51.490742+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:13:51.490742+0000","last_undegraded":"2024-05-09T07:13:51.490742+0000","last_fullsized":"2024-05-09T07:13:51.490742+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491961+0000","last_change":"2024-05-09T07:13:49.938952+0000","last_active":"2024-05-09T07:13:51.491961+0000","last_peered":"2024-05-09T07:13:51.491961+0000","last_clean":"2024-05-09T07:13:51.491961+0000","last_became_active":"2024-05-09T07:13:49.938552+0000","last_became_peered":"2024-05-09T07:13:49.938552+0000","last_unstale":"2024-05-09T07:13:51.491961+0000","last_undegraded":"2024-05-09T07:13:51.491961+0000","last_fullsized":"2024-05-09T07:13:51.491961+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801398+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:13:51.801398+0000","last_peered":"2024-05-09T07:13:51.801398+0000","last_clean":"2024-05-09T07:13:51.801398+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:13:51.801398+0000","last_undegraded":"2024-05-09T07:13:51.801398+0000","last_fullsized":"2024-05-09T07:13:51.801398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697811+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:13:51.697811+0000","last_peered":"2024-05-09T07:13:51.697811+0000","last_clean":"2024-05-09T07:13:51.697811+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:13:51.697811+0000","last_undegraded":"2024-05-09T07:13:51.697811+0000","last_fullsized":"2024-05-09T07:13:51.697811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801577+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:13:51.801577+0000","last_peered":"2024-05-09T07:13:51.801577+0000","last_clean":"2024-05-09T07:13:51.801577+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:13:51.801577+0000","last_undegraded":"2024-05-09T07:13:51.801577+0000","last_fullsized":"2024-05-09T07:13:51.801577+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199329+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:13:52.199329+0000","last_peered":"2024-05-09T07:13:52.199329+0000","last_clean":"2024-05-09T07:13:52.199329+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:13:52.199329+0000","last_undegraded":"2024-05-09T07:13:52.199329+0000","last_fullsized":"2024-05-09T07:13:52.199329+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":26,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202341+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:13:52.202341+0000","last_peered":"2024-05-09T07:13:52.202341+0000","last_clean":"2024-05-09T07:13:52.202341+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:13:52.202341+0000","last_undegraded":"2024-05-09T07:13:52.202341+0000","last_fullsized":"2024-05-09T07:13:52.202341+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800919+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:13:51.800919+0000","last_peered":"2024-05-09T07:13:51.800919+0000","last_clean":"2024-05-09T07:13:51.800919+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:13:51.800919+0000","last_undegraded":"2024-05-09T07:13:51.800919+0000","last_fullsized":"2024-05-09T07:13:51.800919+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492207+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:13:51.492207+0000","last_peered":"2024-05-09T07:13:51.492207+0000","last_clean":"2024-05-09T07:13:51.492207+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:13:51.492207+0000","last_undegraded":"2024-05-09T07:13:51.492207+0000","last_fullsized":"2024-05-09T07:13:51.492207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491259+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:13:51.491259+0000","last_peered":"2024-05-09T07:13:51.491259+0000","last_clean":"2024-05-09T07:13:51.491259+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:13:51.491259+0000","last_undegraded":"2024-05-09T07:13:51.491259+0000","last_fullsized":"2024-05-09T07:13:51.491259+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490763+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:13:51.490763+0000","last_peered":"2024-05-09T07:13:51.490763+0000","last_clean":"2024-05-09T07:13:51.490763+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:13:51.490763+0000","last_undegraded":"2024-05-09T07:13:51.490763+0000","last_fullsized":"2024-05-09T07:13:51.490763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491690+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:13:51.491690+0000","last_peered":"2024-05-09T07:13:51.491690+0000","last_clean":"2024-05-09T07:13:51.491690+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:13:51.491690+0000","last_undegraded":"2024-05-09T07:13:51.491690+0000","last_fullsized":"2024-05-09T07:13:51.491690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698566+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:13:51.698566+0000","last_peered":"2024-05-09T07:13:51.698566+0000","last_clean":"2024-05-09T07:13:51.698566+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:13:51.698566+0000","last_undegraded":"2024-05-09T07:13:51.698566+0000","last_fullsized":"2024-05-09T07:13:51.698566+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198963+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:13:52.198963+0000","last_peered":"2024-05-09T07:13:52.198963+0000","last_clean":"2024-05-09T07:13:52.198963+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:13:52.198963+0000","last_undegraded":"2024-05-09T07:13:52.198963+0000","last_fullsized":"2024-05-09T07:13:52.198963+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202806+0000","last_change":"2024-05-09T07:13:45.484793+0000","last_active":"2024-05-09T07:13:52.202806+0000","last_peered":"2024-05-09T07:13:52.202806+0000","last_clean":"2024-05-09T07:13:52.202806+0000","last_became_active":"2024-05-09T07:13:45.484248+0000","last_became_peered":"2024-05-09T07:13:45.484248+0000","last_unstale":"2024-05-09T07:13:52.202806+0000","last_undegraded":"2024-05-09T07:13:52.202806+0000","last_fullsized":"2024-05-09T07:13:52.202806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491769+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:13:51.491769+0000","last_peered":"2024-05-09T07:13:51.491769+0000","last_clean":"2024-05-09T07:13:51.491769+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:13:51.491769+0000","last_undegraded":"2024-05-09T07:13:51.491769+0000","last_fullsized":"2024-05-09T07:13:51.491769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491114+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:13:51.491114+0000","last_peered":"2024-05-09T07:13:51.491114+0000","last_clean":"2024-05-09T07:13:51.491114+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:13:51.491114+0000","last_undegraded":"2024-05-09T07:13:51.491114+0000","last_fullsized":"2024-05-09T07:13:51.491114+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801532+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:13:51.801532+0000","last_peered":"2024-05-09T07:13:51.801532+0000","last_clean":"2024-05-09T07:13:51.801532+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:13:51.801532+0000","last_undegraded":"2024-05-09T07:13:51.801532+0000","last_fullsized":"2024-05-09T07:13:51.801532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202356+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:13:52.202356+0000","last_peered":"2024-05-09T07:13:52.202356+0000","last_clean":"2024-05-09T07:13:52.202356+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:13:52.202356+0000","last_undegraded":"2024-05-09T07:13:52.202356+0000","last_fullsized":"2024-05-09T07:13:52.202356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491646+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:13:51.491646+0000","last_peered":"2024-05-09T07:13:51.491646+0000","last_clean":"2024-05-09T07:13:51.491646+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:13:51.491646+0000","last_undegraded":"2024-05-09T07:13:51.491646+0000","last_fullsized":"2024-05-09T07:13:51.491646+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490640+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:13:51.490640+0000","last_peered":"2024-05-09T07:13:51.490640+0000","last_clean":"2024-05-09T07:13:51.490640+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:13:51.490640+0000","last_undegraded":"2024-05-09T07:13:51.490640+0000","last_fullsized":"2024-05-09T07:13:51.490640+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490075+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:13:51.490075+0000","last_peered":"2024-05-09T07:13:51.490075+0000","last_clean":"2024-05-09T07:13:51.490075+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:13:51.490075+0000","last_undegraded":"2024-05-09T07:13:51.490075+0000","last_fullsized":"2024-05-09T07:13:51.490075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492028+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:13:51.492028+0000","last_peered":"2024-05-09T07:13:51.492028+0000","last_clean":"2024-05-09T07:13:51.492028+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:13:51.492028+0000","last_undegraded":"2024-05-09T07:13:51.492028+0000","last_fullsized":"2024-05-09T07:13:51.492028+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198795+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:13:52.198795+0000","last_peered":"2024-05-09T07:13:52.198795+0000","last_clean":"2024-05-09T07:13:52.198795+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:13:52.198795+0000","last_undegraded":"2024-05-09T07:13:52.198795+0000","last_fullsized":"2024-05-09T07:13:52.198795+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697830+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:13:51.697830+0000","last_peered":"2024-05-09T07:13:51.697830+0000","last_clean":"2024-05-09T07:13:51.697830+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:13:51.697830+0000","last_undegraded":"2024-05-09T07:13:51.697830+0000","last_fullsized":"2024-05-09T07:13:51.697830+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199305+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:13:52.199305+0000","last_peered":"2024-05-09T07:13:52.199305+0000","last_clean":"2024-05-09T07:13:52.199305+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:13:52.199305+0000","last_undegraded":"2024-05-09T07:13:52.199305+0000","last_fullsized":"2024-05-09T07:13:52.199305+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491484+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:13:51.491484+0000","last_peered":"2024-05-09T07:13:51.491484+0000","last_clean":"2024-05-09T07:13:51.491484+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:13:51.491484+0000","last_undegraded":"2024-05-09T07:13:51.491484+0000","last_fullsized":"2024-05-09T07:13:51.491484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202232+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:13:52.202232+0000","last_peered":"2024-05-09T07:13:52.202232+0000","last_clean":"2024-05-09T07:13:52.202232+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:13:52.202232+0000","last_undegraded":"2024-05-09T07:13:52.202232+0000","last_fullsized":"2024-05-09T07:13:52.202232+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.697989+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:13:51.697989+0000","last_peered":"2024-05-09T07:13:51.697989+0000","last_clean":"2024-05-09T07:13:51.697989+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:13:51.697989+0000","last_undegraded":"2024-05-09T07:13:51.697989+0000","last_fullsized":"2024-05-09T07:13:51.697989+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698344+0000","last_change":"2024-05-09T07:13:45.490251+0000","last_active":"2024-05-09T07:13:51.698344+0000","last_peered":"2024-05-09T07:13:51.698344+0000","last_clean":"2024-05-09T07:13:51.698344+0000","last_became_active":"2024-05-09T07:13:45.489707+0000","last_became_peered":"2024-05-09T07:13:45.489707+0000","last_unstale":"2024-05-09T07:13:51.698344+0000","last_undegraded":"2024-05-09T07:13:51.698344+0000","last_fullsized":"2024-05-09T07:13:51.698344+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490413+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:13:51.490413+0000","last_peered":"2024-05-09T07:13:51.490413+0000","last_clean":"2024-05-09T07:13:51.490413+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:13:51.490413+0000","last_undegraded":"2024-05-09T07:13:51.490413+0000","last_fullsized":"2024-05-09T07:13:51.490413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201756+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:13:52.201756+0000","last_peered":"2024-05-09T07:13:52.201756+0000","last_clean":"2024-05-09T07:13:52.201756+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:13:52.201756+0000","last_undegraded":"2024-05-09T07:13:52.201756+0000","last_fullsized":"2024-05-09T07:13:52.201756+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202663+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:13:52.202663+0000","last_peered":"2024-05-09T07:13:52.202663+0000","last_clean":"2024-05-09T07:13:52.202663+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:13:52.202663+0000","last_undegraded":"2024-05-09T07:13:52.202663+0000","last_fullsized":"2024-05-09T07:13:52.202663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490395+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:13:51.490395+0000","last_peered":"2024-05-09T07:13:51.490395+0000","last_clean":"2024-05-09T07:13:51.490395+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:13:51.490395+0000","last_undegraded":"2024-05-09T07:13:51.490395+0000","last_fullsized":"2024-05-09T07:13:51.490395+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199283+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:13:52.199283+0000","last_peered":"2024-05-09T07:13:52.199283+0000","last_clean":"2024-05-09T07:13:52.199283+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:13:52.199283+0000","last_undegraded":"2024-05-09T07:13:52.199283+0000","last_fullsized":"2024-05-09T07:13:52.199283+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698652+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:13:51.698652+0000","last_peered":"2024-05-09T07:13:51.698652+0000","last_clean":"2024-05-09T07:13:51.698652+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:13:51.698652+0000","last_undegraded":"2024-05-09T07:13:51.698652+0000","last_fullsized":"2024-05-09T07:13:51.698652+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490223+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:13:51.490223+0000","last_peered":"2024-05-09T07:13:51.490223+0000","last_clean":"2024-05-09T07:13:51.490223+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:13:51.490223+0000","last_undegraded":"2024-05-09T07:13:51.490223+0000","last_fullsized":"2024-05-09T07:13:51.490223+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201384+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:13:52.201384+0000","last_peered":"2024-05-09T07:13:52.201384+0000","last_clean":"2024-05-09T07:13:52.201384+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:13:52.201384+0000","last_undegraded":"2024-05-09T07:13:52.201384+0000","last_fullsized":"2024-05-09T07:13:52.201384+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698489+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:13:51.698489+0000","last_peered":"2024-05-09T07:13:51.698489+0000","last_clean":"2024-05-09T07:13:51.698489+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:13:51.698489+0000","last_undegraded":"2024-05-09T07:13:51.698489+0000","last_fullsized":"2024-05-09T07:13:51.698489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801555+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:13:51.801555+0000","last_peered":"2024-05-09T07:13:51.801555+0000","last_clean":"2024-05-09T07:13:51.801555+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:13:51.801555+0000","last_undegraded":"2024-05-09T07:13:51.801555+0000","last_fullsized":"2024-05-09T07:13:51.801555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.800811+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:13:51.800811+0000","last_peered":"2024-05-09T07:13:51.800811+0000","last_clean":"2024-05-09T07:13:51.800811+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:13:51.800811+0000","last_undegraded":"2024-05-09T07:13:51.800811+0000","last_fullsized":"2024-05-09T07:13:51.800811+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698054+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:13:51.698054+0000","last_peered":"2024-05-09T07:13:51.698054+0000","last_clean":"2024-05-09T07:13:51.698054+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:13:51.698054+0000","last_undegraded":"2024-05-09T07:13:51.698054+0000","last_fullsized":"2024-05-09T07:13:51.698054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489838+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:13:51.489838+0000","last_peered":"2024-05-09T07:13:51.489838+0000","last_clean":"2024-05-09T07:13:51.489838+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:13:51.489838+0000","last_undegraded":"2024-05-09T07:13:51.489838+0000","last_fullsized":"2024-05-09T07:13:51.489838+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199184+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:13:52.199184+0000","last_peered":"2024-05-09T07:13:52.199184+0000","last_clean":"2024-05-09T07:13:52.199184+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:13:52.199184+0000","last_undegraded":"2024-05-09T07:13:52.199184+0000","last_fullsized":"2024-05-09T07:13:52.199184+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201678+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:13:52.201678+0000","last_peered":"2024-05-09T07:13:52.201678+0000","last_clean":"2024-05-09T07:13:52.201678+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:13:52.201678+0000","last_undegraded":"2024-05-09T07:13:52.201678+0000","last_fullsized":"2024-05-09T07:13:52.201678+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201895+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:13:52.201895+0000","last_peered":"2024-05-09T07:13:52.201895+0000","last_clean":"2024-05-09T07:13:52.201895+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:13:52.201895+0000","last_undegraded":"2024-05-09T07:13:52.201895+0000","last_fullsized":"2024-05-09T07:13:52.201895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492100+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:13:51.492100+0000","last_peered":"2024-05-09T07:13:51.492100+0000","last_clean":"2024-05-09T07:13:51.492100+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:13:51.492100+0000","last_undegraded":"2024-05-09T07:13:51.492100+0000","last_fullsized":"2024-05-09T07:13:51.492100+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698135+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:13:51.698135+0000","last_peered":"2024-05-09T07:13:51.698135+0000","last_clean":"2024-05-09T07:13:51.698135+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:13:51.698135+0000","last_undegraded":"2024-05-09T07:13:51.698135+0000","last_fullsized":"2024-05-09T07:13:51.698135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202333+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:13:52.202333+0000","last_peered":"2024-05-09T07:13:52.202333+0000","last_clean":"2024-05-09T07:13:52.202333+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:13:52.202333+0000","last_undegraded":"2024-05-09T07:13:52.202333+0000","last_fullsized":"2024-05-09T07:13:52.202333+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491909+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:13:51.491909+0000","last_peered":"2024-05-09T07:13:51.491909+0000","last_clean":"2024-05-09T07:13:51.491909+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:13:51.491909+0000","last_undegraded":"2024-05-09T07:13:51.491909+0000","last_fullsized":"2024-05-09T07:13:51.491909+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198932+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:13:52.198932+0000","last_peered":"2024-05-09T07:13:52.198932+0000","last_clean":"2024-05-09T07:13:52.198932+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:13:52.198932+0000","last_undegraded":"2024-05-09T07:13:52.198932+0000","last_fullsized":"2024-05-09T07:13:52.198932+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199299+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:13:52.199299+0000","last_peered":"2024-05-09T07:13:52.199299+0000","last_clean":"2024-05-09T07:13:52.199299+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:13:52.199299+0000","last_undegraded":"2024-05-09T07:13:52.199299+0000","last_fullsized":"2024-05-09T07:13:52.199299+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202849+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:13:52.202849+0000","last_peered":"2024-05-09T07:13:52.202849+0000","last_clean":"2024-05-09T07:13:52.202849+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:13:52.202849+0000","last_undegraded":"2024-05-09T07:13:52.202849+0000","last_fullsized":"2024-05-09T07:13:52.202849+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698233+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:13:51.698233+0000","last_peered":"2024-05-09T07:13:51.698233+0000","last_clean":"2024-05-09T07:13:51.698233+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:13:51.698233+0000","last_undegraded":"2024-05-09T07:13:51.698233+0000","last_fullsized":"2024-05-09T07:13:51.698233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":33,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:54.939707+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:13:54.939707+0000","last_peered":"2024-05-09T07:13:54.939707+0000","last_clean":"2024-05-09T07:13:54.939707+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:13:54.939707+0000","last_undegraded":"2024-05-09T07:13:54.939707+0000","last_fullsized":"2024-05-09T07:13:54.939707+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492117+0000","last_change":"2024-05-09T07:13:49.503824+0000","last_active":"2024-05-09T07:13:51.492117+0000","last_peered":"2024-05-09T07:13:51.492117+0000","last_clean":"2024-05-09T07:13:51.492117+0000","last_became_active":"2024-05-09T07:13:49.503453+0000","last_became_peered":"2024-05-09T07:13:49.503453+0000","last_unstale":"2024-05-09T07:13:51.492117+0000","last_undegraded":"2024-05-09T07:13:51.492117+0000","last_fullsized":"2024-05-09T07:13:51.492117+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801449+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:13:51.801449+0000","last_peered":"2024-05-09T07:13:51.801449+0000","last_clean":"2024-05-09T07:13:51.801449+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:13:51.801449+0000","last_undegraded":"2024-05-09T07:13:51.801449+0000","last_fullsized":"2024-05-09T07:13:51.801449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490306+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:13:51.490306+0000","last_peered":"2024-05-09T07:13:51.490306+0000","last_clean":"2024-05-09T07:13:51.490306+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:13:51.490306+0000","last_undegraded":"2024-05-09T07:13:51.490306+0000","last_fullsized":"2024-05-09T07:13:51.490306+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801327+0000","last_change":"2024-05-09T07:13:49.509136+0000","last_active":"2024-05-09T07:13:51.801327+0000","last_peered":"2024-05-09T07:13:51.801327+0000","last_clean":"2024-05-09T07:13:51.801327+0000","last_became_active":"2024-05-09T07:13:49.508102+0000","last_became_peered":"2024-05-09T07:13:49.508102+0000","last_unstale":"2024-05-09T07:13:51.801327+0000","last_undegraded":"2024-05-09T07:13:51.801327+0000","last_fullsized":"2024-05-09T07:13:51.801327+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199289+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:13:52.199289+0000","last_peered":"2024-05-09T07:13:52.199289+0000","last_clean":"2024-05-09T07:13:52.199289+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:13:52.199289+0000","last_undegraded":"2024-05-09T07:13:52.199289+0000","last_fullsized":"2024-05-09T07:13:52.199289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202839+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:13:52.202839+0000","last_peered":"2024-05-09T07:13:52.202839+0000","last_clean":"2024-05-09T07:13:52.202839+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:13:52.202839+0000","last_undegraded":"2024-05-09T07:13:52.202839+0000","last_fullsized":"2024-05-09T07:13:52.202839+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492108+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:13:51.492108+0000","last_peered":"2024-05-09T07:13:51.492108+0000","last_clean":"2024-05-09T07:13:51.492108+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:13:51.492108+0000","last_undegraded":"2024-05-09T07:13:51.492108+0000","last_fullsized":"2024-05-09T07:13:51.492108+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202701+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:13:52.202701+0000","last_peered":"2024-05-09T07:13:52.202701+0000","last_clean":"2024-05-09T07:13:52.202701+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:13:52.202701+0000","last_undegraded":"2024-05-09T07:13:52.202701+0000","last_fullsized":"2024-05-09T07:13:52.202701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202142+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:13:52.202142+0000","last_peered":"2024-05-09T07:13:52.202142+0000","last_clean":"2024-05-09T07:13:52.202142+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:13:52.202142+0000","last_undegraded":"2024-05-09T07:13:52.202142+0000","last_fullsized":"2024-05-09T07:13:52.202142+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491246+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:13:51.491246+0000","last_peered":"2024-05-09T07:13:51.491246+0000","last_clean":"2024-05-09T07:13:51.491246+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:13:51.491246+0000","last_undegraded":"2024-05-09T07:13:51.491246+0000","last_fullsized":"2024-05-09T07:13:51.491246+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199132+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:13:52.199132+0000","last_peered":"2024-05-09T07:13:52.199132+0000","last_clean":"2024-05-09T07:13:52.199132+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:13:52.199132+0000","last_undegraded":"2024-05-09T07:13:52.199132+0000","last_fullsized":"2024-05-09T07:13:52.199132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490579+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:13:51.490579+0000","last_peered":"2024-05-09T07:13:51.490579+0000","last_clean":"2024-05-09T07:13:51.490579+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:13:51.490579+0000","last_undegraded":"2024-05-09T07:13:51.490579+0000","last_fullsized":"2024-05-09T07:13:51.490579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201450+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:13:52.201450+0000","last_peered":"2024-05-09T07:13:52.201450+0000","last_clean":"2024-05-09T07:13:52.201450+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:13:52.201450+0000","last_undegraded":"2024-05-09T07:13:52.201450+0000","last_fullsized":"2024-05-09T07:13:52.201450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202518+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:13:52.202518+0000","last_peered":"2024-05-09T07:13:52.202518+0000","last_clean":"2024-05-09T07:13:52.202518+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:13:52.202518+0000","last_undegraded":"2024-05-09T07:13:52.202518+0000","last_fullsized":"2024-05-09T07:13:52.202518+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491038+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:13:51.491038+0000","last_peered":"2024-05-09T07:13:51.491038+0000","last_clean":"2024-05-09T07:13:51.491038+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:13:51.491038+0000","last_undegraded":"2024-05-09T07:13:51.491038+0000","last_fullsized":"2024-05-09T07:13:51.491038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491161+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:13:51.491161+0000","last_peered":"2024-05-09T07:13:51.491161+0000","last_clean":"2024-05-09T07:13:51.491161+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:13:51.491161+0000","last_undegraded":"2024-05-09T07:13:51.491161+0000","last_fullsized":"2024-05-09T07:13:51.491161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201962+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:13:52.201962+0000","last_peered":"2024-05-09T07:13:52.201962+0000","last_clean":"2024-05-09T07:13:52.201962+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:13:52.201962+0000","last_undegraded":"2024-05-09T07:13:52.201962+0000","last_fullsized":"2024-05-09T07:13:52.201962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199171+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:13:52.199171+0000","last_peered":"2024-05-09T07:13:52.199171+0000","last_clean":"2024-05-09T07:13:52.199171+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:13:52.199171+0000","last_undegraded":"2024-05-09T07:13:52.199171+0000","last_fullsized":"2024-05-09T07:13:52.199171+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490242+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:13:51.490242+0000","last_peered":"2024-05-09T07:13:51.490242+0000","last_clean":"2024-05-09T07:13:51.490242+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:13:51.490242+0000","last_undegraded":"2024-05-09T07:13:51.490242+0000","last_fullsized":"2024-05-09T07:13:51.490242+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491319+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:13:51.491319+0000","last_peered":"2024-05-09T07:13:51.491319+0000","last_clean":"2024-05-09T07:13:51.491319+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:13:51.491319+0000","last_undegraded":"2024-05-09T07:13:51.491319+0000","last_fullsized":"2024-05-09T07:13:51.491319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801024+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:13:51.801024+0000","last_peered":"2024-05-09T07:13:51.801024+0000","last_clean":"2024-05-09T07:13:51.801024+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:13:51.801024+0000","last_undegraded":"2024-05-09T07:13:51.801024+0000","last_fullsized":"2024-05-09T07:13:51.801024+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492243+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:13:51.492243+0000","last_peered":"2024-05-09T07:13:51.492243+0000","last_clean":"2024-05-09T07:13:51.492243+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:13:51.492243+0000","last_undegraded":"2024-05-09T07:13:51.492243+0000","last_fullsized":"2024-05-09T07:13:51.492243+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201566+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:13:52.201566+0000","last_peered":"2024-05-09T07:13:52.201566+0000","last_clean":"2024-05-09T07:13:52.201566+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:13:52.201566+0000","last_undegraded":"2024-05-09T07:13:52.201566+0000","last_fullsized":"2024-05-09T07:13:52.201566+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.489979+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:13:51.489979+0000","last_peered":"2024-05-09T07:13:51.489979+0000","last_clean":"2024-05-09T07:13:51.489979+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:13:51.489979+0000","last_undegraded":"2024-05-09T07:13:51.489979+0000","last_fullsized":"2024-05-09T07:13:51.489979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202280+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:13:52.202280+0000","last_peered":"2024-05-09T07:13:52.202280+0000","last_clean":"2024-05-09T07:13:52.202280+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:13:52.202280+0000","last_undegraded":"2024-05-09T07:13:52.202280+0000","last_fullsized":"2024-05-09T07:13:52.202280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199032+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:13:52.199032+0000","last_peered":"2024-05-09T07:13:52.199032+0000","last_clean":"2024-05-09T07:13:52.199032+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:13:52.199032+0000","last_undegraded":"2024-05-09T07:13:52.199032+0000","last_fullsized":"2024-05-09T07:13:52.199032+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491975+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:13:51.491975+0000","last_peered":"2024-05-09T07:13:51.491975+0000","last_clean":"2024-05-09T07:13:51.491975+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:13:51.491975+0000","last_undegraded":"2024-05-09T07:13:51.491975+0000","last_fullsized":"2024-05-09T07:13:51.491975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490135+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:13:51.490135+0000","last_peered":"2024-05-09T07:13:51.490135+0000","last_clean":"2024-05-09T07:13:51.490135+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:13:51.490135+0000","last_undegraded":"2024-05-09T07:13:51.490135+0000","last_fullsized":"2024-05-09T07:13:51.490135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202331+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:13:52.202331+0000","last_peered":"2024-05-09T07:13:52.202331+0000","last_clean":"2024-05-09T07:13:52.202331+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:13:52.202331+0000","last_undegraded":"2024-05-09T07:13:52.202331+0000","last_fullsized":"2024-05-09T07:13:52.202331+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.492151+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:13:51.492151+0000","last_peered":"2024-05-09T07:13:51.492151+0000","last_clean":"2024-05-09T07:13:51.492151+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:13:51.492151+0000","last_undegraded":"2024-05-09T07:13:51.492151+0000","last_fullsized":"2024-05-09T07:13:51.492151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201443+0000","last_change":"2024-05-09T07:13:45.483771+0000","last_active":"2024-05-09T07:13:52.201443+0000","last_peered":"2024-05-09T07:13:52.201443+0000","last_clean":"2024-05-09T07:13:52.201443+0000","last_became_active":"2024-05-09T07:13:45.483484+0000","last_became_peered":"2024-05-09T07:13:45.483484+0000","last_unstale":"2024-05-09T07:13:52.201443+0000","last_undegraded":"2024-05-09T07:13:52.201443+0000","last_fullsized":"2024-05-09T07:13:52.201443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491076+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:13:51.491076+0000","last_peered":"2024-05-09T07:13:51.491076+0000","last_clean":"2024-05-09T07:13:51.491076+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:13:51.491076+0000","last_undegraded":"2024-05-09T07:13:51.491076+0000","last_fullsized":"2024-05-09T07:13:51.491076+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202200+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:13:52.202200+0000","last_peered":"2024-05-09T07:13:52.202200+0000","last_clean":"2024-05-09T07:13:52.202200+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:13:52.202200+0000","last_undegraded":"2024-05-09T07:13:52.202200+0000","last_fullsized":"2024-05-09T07:13:52.202200+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198920+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:13:52.198920+0000","last_peered":"2024-05-09T07:13:52.198920+0000","last_clean":"2024-05-09T07:13:52.198920+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:13:52.198920+0000","last_undegraded":"2024-05-09T07:13:52.198920+0000","last_fullsized":"2024-05-09T07:13:52.198920+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201917+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:13:52.201917+0000","last_peered":"2024-05-09T07:13:52.201917+0000","last_clean":"2024-05-09T07:13:52.201917+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:13:52.201917+0000","last_undegraded":"2024-05-09T07:13:52.201917+0000","last_fullsized":"2024-05-09T07:13:52.201917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698479+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:13:51.698479+0000","last_peered":"2024-05-09T07:13:51.698479+0000","last_clean":"2024-05-09T07:13:51.698479+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:13:51.698479+0000","last_undegraded":"2024-05-09T07:13:51.698479+0000","last_fullsized":"2024-05-09T07:13:51.698479+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490136+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:13:51.490136+0000","last_peered":"2024-05-09T07:13:51.490136+0000","last_clean":"2024-05-09T07:13:51.490136+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:13:51.490136+0000","last_undegraded":"2024-05-09T07:13:51.490136+0000","last_fullsized":"2024-05-09T07:13:51.490136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202983+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:13:52.202983+0000","last_peered":"2024-05-09T07:13:52.202983+0000","last_clean":"2024-05-09T07:13:52.202983+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:13:52.202983+0000","last_undegraded":"2024-05-09T07:13:52.202983+0000","last_fullsized":"2024-05-09T07:13:52.202983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.201343+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:13:52.201343+0000","last_peered":"2024-05-09T07:13:52.201343+0000","last_clean":"2024-05-09T07:13:52.201343+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:13:52.201343+0000","last_undegraded":"2024-05-09T07:13:52.201343+0000","last_fullsized":"2024-05-09T07:13:52.201343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490661+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:13:51.490661+0000","last_peered":"2024-05-09T07:13:51.490661+0000","last_clean":"2024-05-09T07:13:51.490661+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:13:51.490661+0000","last_undegraded":"2024-05-09T07:13:51.490661+0000","last_fullsized":"2024-05-09T07:13:51.490661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.198914+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:13:52.198914+0000","last_peered":"2024-05-09T07:13:52.198914+0000","last_clean":"2024-05-09T07:13:52.198914+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:13:52.198914+0000","last_undegraded":"2024-05-09T07:13:52.198914+0000","last_fullsized":"2024-05-09T07:13:52.198914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491230+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:13:51.491230+0000","last_peered":"2024-05-09T07:13:51.491230+0000","last_clean":"2024-05-09T07:13:51.491230+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:13:51.491230+0000","last_undegraded":"2024-05-09T07:13:51.491230+0000","last_fullsized":"2024-05-09T07:13:51.491230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.801266+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:13:51.801266+0000","last_peered":"2024-05-09T07:13:51.801266+0000","last_clean":"2024-05-09T07:13:51.801266+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:13:51.801266+0000","last_undegraded":"2024-05-09T07:13:51.801266+0000","last_fullsized":"2024-05-09T07:13:51.801266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202141+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:13:52.202141+0000","last_peered":"2024-05-09T07:13:52.202141+0000","last_clean":"2024-05-09T07:13:52.202141+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:13:52.202141+0000","last_undegraded":"2024-05-09T07:13:52.202141+0000","last_fullsized":"2024-05-09T07:13:52.202141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202227+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:13:52.202227+0000","last_peered":"2024-05-09T07:13:52.202227+0000","last_clean":"2024-05-09T07:13:52.202227+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:13:52.202227+0000","last_undegraded":"2024-05-09T07:13:52.202227+0000","last_fullsized":"2024-05-09T07:13:52.202227+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698631+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:13:51.698631+0000","last_peered":"2024-05-09T07:13:51.698631+0000","last_clean":"2024-05-09T07:13:51.698631+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:13:51.698631+0000","last_undegraded":"2024-05-09T07:13:51.698631+0000","last_fullsized":"2024-05-09T07:13:51.698631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.490881+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:13:51.490881+0000","last_peered":"2024-05-09T07:13:51.490881+0000","last_clean":"2024-05-09T07:13:51.490881+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:13:51.490881+0000","last_undegraded":"2024-05-09T07:13:51.490881+0000","last_fullsized":"2024-05-09T07:13:51.490881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698046+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:13:51.698046+0000","last_peered":"2024-05-09T07:13:51.698046+0000","last_clean":"2024-05-09T07:13:51.698046+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:13:51.698046+0000","last_undegraded":"2024-05-09T07:13:51.698046+0000","last_fullsized":"2024-05-09T07:13:51.698046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202156+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:13:52.202156+0000","last_peered":"2024-05-09T07:13:52.202156+0000","last_clean":"2024-05-09T07:13:52.202156+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:13:52.202156+0000","last_undegraded":"2024-05-09T07:13:52.202156+0000","last_fullsized":"2024-05-09T07:13:52.202156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698092+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:13:51.698092+0000","last_peered":"2024-05-09T07:13:51.698092+0000","last_clean":"2024-05-09T07:13:51.698092+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:13:51.698092+0000","last_undegraded":"2024-05-09T07:13:51.698092+0000","last_fullsized":"2024-05-09T07:13:51.698092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.202561+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:13:52.202561+0000","last_peered":"2024-05-09T07:13:52.202561+0000","last_clean":"2024-05-09T07:13:52.202561+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:13:52.202561+0000","last_undegraded":"2024-05-09T07:13:52.202561+0000","last_fullsized":"2024-05-09T07:13:52.202561+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:52.199052+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:13:52.199052+0000","last_peered":"2024-05-09T07:13:52.199052+0000","last_clean":"2024-05-09T07:13:52.199052+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:13:52.199052+0000","last_undegraded":"2024-05-09T07:13:52.199052+0000","last_fullsized":"2024-05-09T07:13:52.199052+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.491163+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:13:51.491163+0000","last_peered":"2024-05-09T07:13:51.491163+0000","last_clean":"2024-05-09T07:13:51.491163+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:13:51.491163+0000","last_undegraded":"2024-05-09T07:13:51.491163+0000","last_fullsized":"2024-05-09T07:13:51.491163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-05-09T07:13:51.698522+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:13:51.698522+0000","last_peered":"2024-05-09T07:13:51.698522+0000","last_clean":"2024-05-09T07:13:51.698522+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:13:51.698522+0000","last_undegraded":"2024-05-09T07:13:51.698522+0000","last_fullsized":"2024-05-09T07:13:51.698522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":864256,"data_stored":698343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":270336,"data_stored":107483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":278528,"data_stored":107519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":272,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":278528,"data_stored":111235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":107393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":107553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":698399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:14:07.234 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T07:14:07.234 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T07:14:07.235 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-09T07:14:07.235 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-09T07:14:07.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.367+0000 7f4a5259e640 1 Processor -- start 2024-05-09T07:14:07.369 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.367+0000 7f4a5259e640 1 -- start start 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a5259e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a9e40 0x7f4a4c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a5259e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a4c0a6170 0x7f4a4c152720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a5259e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a4c1559a0 con 0x7f4a4c14f7e0 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a5259e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4c155750 con 0x7f4a4c0a6170 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a5259e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f4a4c155c80 con 0x7f4a4c0a9e40 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a50b3b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a4c0a6170 0x7f4a4c152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a50b3b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f4a4c0a6170 0x7f4a4c152720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44310/0 (socket says 172.21.15.160:44310) 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a50b3b640 1 -- 172.21.15.160:0/2226591394 learned_addr learned my addr 172.21.15.160:0/2226591394 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.368+0000 7f4a4b7fe640 1 --2- 172.21.15.160:0/2226591394 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a9e40 0x7f4a4c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a50b3b640 1 -- 172.21.15.160:0/2226591394 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a9e40 msgr2=0x7f4a4c1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a50b3b640 1 --2- 172.21.15.160:0/2226591394 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a9e40 0x7f4a4c1521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a50b3b640 1 -- 172.21.15.160:0/2226591394 >> v1:172.21.15.188:6789/0 conn(0x7f4a4c14f7e0 legacy=0x7f4a4c0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a50b3b640 1 -- 172.21.15.160:0/2226591394 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c155f60 con 0x7f4a4c0a6170 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a4b7fe640 1 --2- 172.21.15.160:0/2226591394 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a9e40 0x7f4a4c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a50b3b640 1 --2- 172.21.15.160:0/2226591394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c0a6170 0x7f4a4c152720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06c260 tx=0x7f4a4402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccc6bc64e411d71 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a4affd640 1 -- 172.21.15.160:0/2226591394 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44036070 con 0x7f4a4c0a6170 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a4affd640 1 -- 172.21.15.160:0/2226591394 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44031600 con 0x7f4a4c0a6170 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.369+0000 7f4a4affd640 1 -- 172.21.15.160:0/2226591394 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440319d0 con 0x7f4a4c0a6170 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 -- 172.21.15.160:0/2226591394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c0a6170 msgr2=0x7f4a4c152720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.373 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 --2- 172.21.15.160:0/2226591394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c0a6170 0x7f4a4c152720 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06c260 tx=0x7f4a4402a6a0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 -- 172.21.15.160:0/2226591394 shutdown_connections 2024-05-09T07:14:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 --2- 172.21.15.160:0/2226591394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c0a6170 0x7f4a4c152720 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 --2- 172.21.15.160:0/2226591394 >> v2:172.21.15.188:3300/0 conn(0x7f4a4c0a9e40 0x7f4a4c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 -- 172.21.15.160:0/2226591394 >> 172.21.15.160:0/2226591394 conn(0x7f4a4c0a91e0 msgr2=0x7f4a4c0c33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 -- 172.21.15.160:0/2226591394 shutdown_connections 2024-05-09T07:14:07.374 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.370+0000 7f4a5259e640 1 -- 172.21.15.160:0/2226591394 wait complete. 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.371+0000 7f4a5259e640 1 Processor -- start 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.371+0000 7f4a5259e640 1 -- start start 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a5259e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a6170 0x7f4a4c1ec430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a5259e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c14f7e0 0x7f4a4c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a5259e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 0x7f4a4c146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a5259e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c0c5ff0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a5259e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c14fbc0 con 0x7f4a4c14f7e0 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a5259e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f4a4c0c6170 con 0x7f4a4c0a6170 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 0x7f4a4c146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 0x7f4a4c146a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44316/0 (socket says 172.21.15.160:44316) 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 -- 172.21.15.160:0/1794399233 learned_addr learned my addr 172.21.15.160:0/1794399233 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a4b7fe640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c14f7e0 0x7f4a4c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a4bfff640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a6170 0x7f4a4c1ec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.377 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 -- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a6170 msgr2=0x7f4a4c1ec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a6170 0x7f4a4c1ec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 -- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c14f7e0 msgr2=0x7f4a4c14d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c14f7e0 0x7f4a4c14d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a50b3b640 1 -- 172.21.15.160:0/1794399233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c0a75a0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a4b7fe640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c14f7e0 0x7f4a4c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.372+0000 7f4a4bfff640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a6170 0x7f4a4c1ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.373+0000 7f4a50b3b640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 0x7f4a4c146a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4a4402ac90 tx=0x7f4a44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.373+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44036050 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.373+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a44042070 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.373+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44009510 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.373+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c149150 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.378 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.373+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1474e0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.374+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c075650 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.374+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f4a44007bd0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.375+0000 7f4a48ff9640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4a3806be30 0x7f4a3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.375+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4a440ac0a0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.375+0000 7f4a4bfff640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4a3806be30 0x7f4a3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.379 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.375+0000 7f4a4bfff640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4a3806be30 0x7f4a3806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0015e0 tx=0x7f4a3c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.380 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.379+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4a4407a7a0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.590+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f4a4c0c1a50 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.593 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.591+0000 7f4a48ff9640 1 -- 172.21.15.160:0/1794399233 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f4a4407a0a0 con 0x7f4a4c14e3a0 2024-05-09T07:14:07.593 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:07.593 INFO:teuthology.orchestra.run.smithi160.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4a3806be30 msgr2=0x7f4a3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4a3806be30 0x7f4a3806e2f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c0015e0 tx=0x7f4a3c009040 comp rx=0 tx=0).stop 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 msgr2=0x7f4a4c146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 0x7f4a4c146a50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4a4402ac90 tx=0x7f4a44004a40 comp rx=0 tx=0).stop 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 shutdown_connections 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f4a3806be30 0x7f4a3806e2f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f4a4c14f7e0 0x7f4a4c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4a4c14e3a0 0x7f4a4c146a50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 --2- 172.21.15.160:0/1794399233 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f4a4c0a6170 0x7f4a4c1ec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 >> 172.21.15.160:0/1794399233 conn(0x7f4a4c0a91e0 msgr2=0x7f4a4c0a7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:07.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 shutdown_connections 2024-05-09T07:14:07.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.593+0000 7f4a5259e640 1 -- 172.21.15.160:0/1794399233 wait complete. 2024-05-09T07:14:07.606 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-09T07:14:07.606 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:14:07.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.704+0000 7f3bc9c7d640 1 Processor -- start 2024-05-09T07:14:07.706 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.704+0000 7f3bc9c7d640 1 -- start start 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc9c7d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3bc4153300 0x7f3bc40c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc9c7d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3bc40a5fb0 0x7f3bc40bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc9c7d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3bc41563b0 con 0x7f3bc40c5e20 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc9c7d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc40c1630 con 0x7f3bc40a5fb0 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc9c7d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3bc40c19d0 con 0x7f3bc4153300 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc2ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3bc40a5fb0 0x7f3bc40bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f3bc40c5e20 0x7f3bc40bc360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36424/0 (socket says 172.21.15.160:36424) 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc3fff640 1 -- 172.21.15.160:0/1229410799 learned_addr learned my addr 172.21.15.160:0/1229410799 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc37fe640 1 --2- 172.21.15.160:0/1229410799 >> v2:172.21.15.188:3300/0 conn(0x7f3bc4153300 0x7f3bc40c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/1229410799 >> v2:172.21.15.188:3300/0 conn(0x7f3bc4153300 msgr2=0x7f3bc40c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc2ffd640 1 --2- 172.21.15.160:0/1229410799 >> v2:172.21.15.188:3300/0 conn(0x7f3bc4153300 0x7f3bc40c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/1229410799 >> v1:172.21.15.160:6789/0 conn(0x7f3bc40c5e20 legacy=0x7f3bc40bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/1229410799 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc4156690 con 0x7f3bc40a5fb0 2024-05-09T07:14:07.708 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.705+0000 7f3bc27fc640 1 -- 172.21.15.160:0/1229410799 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3461186749 0 0) 0x7f3bc41563b0 con 0x7f3bc40c5e20 2024-05-09T07:14:07.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.706+0000 7f3bc2ffd640 1 --2- 172.21.15.160:0/1229410799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc40bbe20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3bc406c1d0 tx=0x7f3bb802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29484fcff357ce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.706+0000 7f3bc27fc640 1 -- 172.21.15.160:0/1229410799 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8007740 con 0x7f3bc40a5fb0 2024-05-09T07:14:07.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.706+0000 7f3bc27fc640 1 -- 172.21.15.160:0/1229410799 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb80078e0 con 0x7f3bc40a5fb0 2024-05-09T07:14:07.709 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.706+0000 7f3bc27fc640 1 -- 172.21.15.160:0/1229410799 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8007cb0 con 0x7f3bc40a5fb0 2024-05-09T07:14:07.711 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/1229410799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40a5fb0 msgr2=0x7f3bc40bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/1229410799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc40bbe20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3bc406c1d0 tx=0x7f3bb802a680 comp rx=0 tx=0).stop 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/1229410799 shutdown_connections 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/1229410799 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc40bbe20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/1229410799 >> v2:172.21.15.188:3300/0 conn(0x7f3bc4153300 0x7f3bc40c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/1229410799 >> 172.21.15.160:0/1229410799 conn(0x7f3bc40c7580 msgr2=0x7f3bc40a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/1229410799 shutdown_connections 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.707+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/1229410799 wait complete. 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.708+0000 7f3bc9c7d640 1 Processor -- start 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.708+0000 7f3bc9c7d640 1 -- start start 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc41e0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 0x7f3bc4151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bc4153300 0x7f3bc40bf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40c2420 con 0x7f3bc40c5e20 2024-05-09T07:14:07.712 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3bc40c1f00 con 0x7f3bc40a5fb0 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3bc40bd220 con 0x7f3bc4153300 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 0x7f3bc4151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 0x7f3bc4151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44336/0 (socket says 172.21.15.160:44336) 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/237128644 learned_addr learned my addr 172.21.15.160:0/237128644 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc37fe640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc41e0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc3fff640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bc4153300 0x7f3bc40bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bc4153300 msgr2=0x7f3bc40bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bc4153300 0x7f3bc40bf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc40a5fb0 msgr2=0x7f3bc41e0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc41e0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 -- 172.21.15.160:0/237128644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bc40c2c60 con 0x7f3bc40c5e20 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc2ffd640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 0x7f3bc4151b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3bb802ac70 tx=0x7f3bb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb803f070 con 0x7f3bc40c5e20 2024-05-09T07:14:07.713 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bb8031c40 con 0x7f3bc40c5e20 2024-05-09T07:14:07.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bc41ed320 con 0x7f3bc40c5e20 2024-05-09T07:14:07.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8044460 con 0x7f3bc40c5e20 2024-05-09T07:14:07.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.709+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bc41e9590 con 0x7f3bc40c5e20 2024-05-09T07:14:07.714 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.710+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bc41e90c0 con 0x7f3bc40c5e20 2024-05-09T07:14:07.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.711+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3bb803b410 con 0x7f3bc40c5e20 2024-05-09T07:14:07.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.711+0000 7f3babfff640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bb006be60 0x7f3bb006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:07.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.711+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3bb80ac7c0 con 0x7f3bc40c5e20 2024-05-09T07:14:07.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.711+0000 7f3bc37fe640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bb006be60 0x7f3bb006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:07.715 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.712+0000 7f3bc37fe640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bb006be60 0x7f3bb006e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:07.717 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.716+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3bb807a870 con 0x7f3bc40c5e20 2024-05-09T07:14:07.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.890+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3bc40756c0 con 0x7f3bc40c5e20 2024-05-09T07:14:07.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.891+0000 7f3babfff640 1 -- 172.21.15.160:0/237128644 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f3bb807eb70 con 0x7f3bc40c5e20 2024-05-09T07:14:07.893 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:07.894 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bb006be60 msgr2=0x7f3bb006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bb006be60 0x7f3bb006e320 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4006fd0 tx=0x7f3bb4008040 comp rx=0 tx=0).stop 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 msgr2=0x7f3bc4151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 0x7f3bc4151b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3bb802ac70 tx=0x7f3bb8004a40 comp rx=0 tx=0).stop 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 shutdown_connections 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bb006be60 0x7f3bb006e320 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3bc4153300 0x7f3bc40bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3bc40c5e20 0x7f3bc4151b40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 --2- 172.21.15.160:0/237128644 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3bc40a5fb0 0x7f3bc41e0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 >> 172.21.15.160:0/237128644 conn(0x7f3bc40c7580 msgr2=0x7f3bc40c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:07.896 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 shutdown_connections 2024-05-09T07:14:07.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:07.894+0000 7f3bc9c7d640 1 -- 172.21.15.160:0/237128644 wait complete. 2024-05-09T07:14:07.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:14:07.908 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:14:08.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.009+0000 7f8ebe6fc640 1 Processor -- start 2024-05-09T07:14:08.011 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.009+0000 7f8ebe6fc640 1 -- start start 2024-05-09T07:14:08.013 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8ebe6fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8eb8071f80 0x7f8eb80c6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8ebe6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb806b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8ebe6fc640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8eb81595e0 con 0x7f8eb806b160 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8ebe6fc640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f8eb80730a0 con 0x7f8eb8071f80 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8ebe6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb8073660 con 0x7f8eb80719e0 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb77fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8eb8071f80 0x7f8eb80c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb7fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8eb806b160 0x7f8eb8155540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36434/0 (socket says 172.21.15.160:36434) 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb77fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f8eb8071f80 0x7f8eb80c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44348/0 (socket says 172.21.15.160:44348) 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb7fff640 1 -- 172.21.15.160:0/438615524 learned_addr learned my addr 172.21.15.160:0/438615524 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8ebcc99640 1 --2- 172.21.15.160:0/438615524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb806b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.014 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb77fe640 1 -- 172.21.15.160:0/438615524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 msgr2=0x7f8eb806b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb77fe640 1 --2- 172.21.15.160:0/438615524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb806b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb77fe640 1 -- 172.21.15.160:0/438615524 >> v1:172.21.15.160:6789/0 conn(0x7f8eb806b160 legacy=0x7f8eb8155540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb77fe640 1 -- 172.21.15.160:0/438615524 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb81598c0 con 0x7f8eb8071f80 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.010+0000 7f8eb6ffd640 1 -- 172.21.15.160:0/438615524 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1242106959 0 0) 0x7f8eb81595e0 con 0x7f8eb806b160 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.011+0000 7f8eb77fe640 1 --2- 172.21.15.160:0/438615524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb8071f80 0x7f8eb80c6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0097a0 tx=0x7f8eac02a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6c5d363508a4bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.011+0000 7f8eb6ffd640 1 -- 172.21.15.160:0/438615524 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac037070 con 0x7f8eb8071f80 2024-05-09T07:14:08.015 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.011+0000 7f8eb6ffd640 1 -- 172.21.15.160:0/438615524 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eac02adb0 con 0x7f8eb8071f80 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.011+0000 7f8eb6ffd640 1 -- 172.21.15.160:0/438615524 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8eac032660 con 0x7f8eb8071f80 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/438615524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb8071f80 msgr2=0x7f8eb80c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/438615524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb8071f80 0x7f8eb80c6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8eac0097a0 tx=0x7f8eac02a060 comp rx=0 tx=0).stop 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/438615524 shutdown_connections 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/438615524 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb806b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/438615524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb8071f80 0x7f8eb80c6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/438615524 >> 172.21.15.160:0/438615524 conn(0x7f8eb80a98a0 msgr2=0x7f8eb80a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/438615524 shutdown_connections 2024-05-09T07:14:08.016 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.012+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/438615524 wait complete. 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.013+0000 7f8ebe6fc640 1 Processor -- start 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.013+0000 7f8ebe6fc640 1 -- start start 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 0x7f8eb80c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb80c5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eb8071f80 0x7f8eb80ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8152e30 con 0x7f8eb806b160 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb8155c50 con 0x7f8eb8071f80 2024-05-09T07:14:08.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb8152fb0 con 0x7f8eb80719e0 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 0x7f8eb80c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 0x7f8eb80c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44356/0 (socket says 172.21.15.160:44356) 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 -- 172.21.15.160:0/3607554848 learned_addr learned my addr 172.21.15.160:0/3607554848 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb77fe640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb80c5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 -- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 msgr2=0x7f8eb80c5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb80c5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebcc99640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eb8071f80 0x7f8eb80ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 -- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eb8071f80 msgr2=0x7f8eb80ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eb8071f80 0x7f8eb80ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 -- 172.21.15.160:0/3607554848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb8156ad0 con 0x7f8eb806b160 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb7fff640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 0x7f8eb80c4b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c4d0 tx=0x7f8ea800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8017070 con 0x7f8eb806b160 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ea8005d00 con 0x7f8eb806b160 2024-05-09T07:14:08.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb8156db0 con 0x7f8eb806b160 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80072b0 con 0x7f8eb806b160 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.014+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb8157370 con 0x7f8eb806b160 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.016+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8eb8075650 con 0x7f8eb806b160 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.016+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8ea801a070 con 0x7f8eb806b160 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.016+0000 7f8eb4ff9640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8ea406be60 0x7f8ea406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.017+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8ea808a540 con 0x7f8eb806b160 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.017+0000 7f8eb77fe640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8ea406be60 0x7f8ea406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.017+0000 7f8eb77fe640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8ea406be60 0x7f8ea406e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8eac034000 tx=0x7f8eac02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.020+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8ea8058bb0 con 0x7f8eb806b160 2024-05-09T07:14:08.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.191+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8eb8072e80 con 0x7f8eb806b160 2024-05-09T07:14:08.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.192+0000 7f8eb4ff9640 1 -- 172.21.15.160:0/3607554848 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7f8ea80584b0 con 0x7f8eb806b160 2024-05-09T07:14:08.195 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:14:08.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8ea406be60 msgr2=0x7f8ea406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8ea406be60 0x7f8ea406e320 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8eac034000 tx=0x7f8eac02e000 comp rx=0 tx=0).stop 2024-05-09T07:14:08.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 msgr2=0x7f8eb80c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 0x7f8eb80c4b70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8ea800c4d0 tx=0x7f8ea800c990 comp rx=0 tx=0).stop 2024-05-09T07:14:08.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 shutdown_connections 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8ea406be60 0x7f8ea406e320 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8eb8071f80 0x7f8eb80ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8eb80719e0 0x7f8eb80c5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 --2- 172.21.15.160:0/3607554848 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8eb806b160 0x7f8eb80c4b70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 >> 172.21.15.160:0/3607554848 conn(0x7f8eb80a98a0 msgr2=0x7f8eb8153d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 shutdown_connections 2024-05-09T07:14:08.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.194+0000 7f8ebe6fc640 1 -- 172.21.15.160:0/3607554848 wait complete. 2024-05-09T07:14:08.208 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:14:08.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.308+0000 7fd8cacae640 1 Processor -- start 2024-05-09T07:14:08.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.308+0000 7fd8cacae640 1 -- start start 2024-05-09T07:14:08.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8cacae640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8c41508f0 0x7fd8c40abb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8cacae640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c40a9c00 con 0x7fd8c40c2ef0 2024-05-09T07:14:08.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8cacae640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd8c4153fd0 con 0x7fd8c40ac040 2024-05-09T07:14:08.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8cacae640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fd8c40731e0 con 0x7fd8c41508f0 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c8a4a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fd8c41508f0 0x7fd8c40abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c924b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fd8c40c2ef0 0x7fd8c415db60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36442/0 (socket says 172.21.15.160:36442) 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c924b640 1 -- 172.21.15.160:0/201594075 learned_addr learned my addr 172.21.15.160:0/201594075 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/201594075 >> v1:172.21.15.188:6789/0 conn(0x7fd8c40ac040 legacy=0x7fd8c40c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/201594075 >> v1:172.21.15.160:6789/0 conn(0x7fd8c40c2ef0 legacy=0x7fd8c415db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/201594075 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c40a9920 con 0x7fd8c41508f0 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.309+0000 7fd8c37fe640 1 -- 172.21.15.160:0/201594075 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4001446931 0 0) 0x7fd8c40a9c00 con 0x7fd8c40c2ef0 2024-05-09T07:14:08.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.310+0000 7fd8c8a4a640 1 --2- 172.21.15.160:0/201594075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c41508f0 0x7fd8c40abb00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097a0 tx=0x7fd8b802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f7b42828ccebce server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.310+0000 7fd8c37fe640 1 -- 172.21.15.160:0/201594075 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8037070 con 0x7fd8c41508f0 2024-05-09T07:14:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.310+0000 7fd8c37fe640 1 -- 172.21.15.160:0/201594075 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b8032440 con 0x7fd8c41508f0 2024-05-09T07:14:08.314 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.310+0000 7fd8c37fe640 1 -- 172.21.15.160:0/201594075 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8032810 con 0x7fd8c41508f0 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 -- 172.21.15.160:0/201594075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c41508f0 msgr2=0x7fd8c40abb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 --2- 172.21.15.160:0/201594075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c41508f0 0x7fd8c40abb00 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097a0 tx=0x7fd8b802a1e0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 -- 172.21.15.160:0/201594075 shutdown_connections 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 --2- 172.21.15.160:0/201594075 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c41508f0 0x7fd8c40abb00 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 -- 172.21.15.160:0/201594075 >> 172.21.15.160:0/201594075 conn(0x7fd8c40c38d0 msgr2=0x7fd8c40c3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 -- 172.21.15.160:0/201594075 shutdown_connections 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.311+0000 7fd8cacae640 1 -- 172.21.15.160:0/201594075 wait complete. 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.312+0000 7fd8cacae640 1 Processor -- start 2024-05-09T07:14:08.315 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.312+0000 7fd8cacae640 1 -- start start 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 0x7fd8c4147070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c40c2ef0 0x7fd8c40c0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c41508f0 0x7fd8c41eb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c40aa000 con 0x7fd8c40ac040 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c40a96d0 con 0x7fd8c40c2ef0 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c4153c60 con 0x7fd8c41508f0 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 0x7fd8c4147070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 0x7fd8c4147070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44370/0 (socket says 172.21.15.160:44370) 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/2512803921 learned_addr learned my addr 172.21.15.160:0/2512803921 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c924b640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c41508f0 0x7fd8c41eb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c3fff640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c40c2ef0 0x7fd8c40c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c41508f0 msgr2=0x7fd8c41eb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c41508f0 0x7fd8c41eb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c40c2ef0 msgr2=0x7fd8c40c0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c40c2ef0 0x7fd8c40c0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 -- 172.21.15.160:0/2512803921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c41eae40 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c8a4a640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 0x7fd8c4147070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097a0 tx=0x7fd8b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8041070 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd8b803c870 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c414cab0 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8037040 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.313+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c414c210 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.314+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c4075650 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.315+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd8b8044070 con 0x7fd8c40ac040 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.315+0000 7fd8c17fa640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd8a806be30 0x7fd8a806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.319 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.315+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fd8b80abfd0 con 0x7fd8c40ac040 2024-05-09T07:14:08.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.315+0000 7fd8c3fff640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd8a806be30 0x7fd8a806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.316+0000 7fd8c3fff640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd8a806be30 0x7fd8a806e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0007fd0 tx=0x7fd8b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.319+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fd8b807a7f0 con 0x7fd8c40ac040 2024-05-09T07:14:08.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.494+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fd8c40c45e0 con 0x7fd8c40ac040 2024-05-09T07:14:08.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.494+0000 7fd8c17fa640 1 -- 172.21.15.160:0/2512803921 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fd8b807a0f0 con 0x7fd8c40ac040 2024-05-09T07:14:08.497 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:14:08.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd8a806be30 msgr2=0x7fd8a806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd8a806be30 0x7fd8a806e2f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0007fd0 tx=0x7fd8b0009040 comp rx=0 tx=0).stop 2024-05-09T07:14:08.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 msgr2=0x7fd8c4147070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 0x7fd8c4147070 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80097a0 tx=0x7fd8b80046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 shutdown_connections 2024-05-09T07:14:08.499 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd8a806be30 0x7fd8a806e2f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd8c40c2ef0 0x7fd8c40c0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd8c41508f0 0x7fd8c41eb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 --2- 172.21.15.160:0/2512803921 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd8c40ac040 0x7fd8c4147070 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 >> 172.21.15.160:0/2512803921 conn(0x7fd8c40c38d0 msgr2=0x7fd8c40ac770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 shutdown_connections 2024-05-09T07:14:08.500 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.496+0000 7fd8cacae640 1 -- 172.21.15.160:0/2512803921 wait complete. 2024-05-09T07:14:08.510 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:14:08.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.610+0000 7f204cf61640 1 Processor -- start 2024-05-09T07:14:08.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.610+0000 7f204cf61640 1 -- start start 2024-05-09T07:14:08.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f204cf61640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f20481573a0 0x7f20480c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f204cf61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 0x7f204815dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f204cf61640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20480c7620 con 0x7f204806fac0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f204cf61640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f20480714e0 con 0x7f20481573a0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f204cf61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f20480c78c0 con 0x7f20480c4690 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f203dd9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f204806fac0 0x7f20480c3f80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36444/0 (socket says 172.21.15.160:36444) 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f203dd9b640 1 -- 172.21.15.160:0/2043986407 learned_addr learned my addr 172.21.15.160:0/2043986407 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f204659c640 1 --2- 172.21.15.160:0/2043986407 >> v2:172.21.15.188:3300/0 conn(0x7f20481573a0 0x7f20480c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f2046d9d640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 0x7f204815dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1886705176 0 0) 0x7f20480c7620 con 0x7f204806fac0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.611+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2034003520 con 0x7f204806fac0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2360266413 0 0) 0x7f2034003520 con 0x7f204806fac0 2024-05-09T07:14:08.617 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 msgr2=0x7f204815dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 0x7f204815dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 >> v2:172.21.15.188:3300/0 conn(0x7f20481573a0 msgr2=0x7f20480c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 --2- 172.21.15.160:0/2043986407 >> v2:172.21.15.188:3300/0 conn(0x7f20481573a0 0x7f20480c8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2048071200 con 0x7f204806fac0 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f2030001670 con 0x7f204806fac0 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 >> v1:172.21.15.160:6789/0 conn(0x7f204806fac0 legacy=0x7f20480c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 0x7f20340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034003f30 0x7f2034004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2034004d40 0x7f20340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f20480714e0 con 0x7f2034004d40 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20480c78c0 con 0x7f2034003f30 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f203400b9f0 con 0x7f20480c4690 2024-05-09T07:14:08.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2046d9d640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2034004d40 0x7f20340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f204659c640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 0x7f20340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f203dd9b640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034003f30 0x7f2034004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2046d9d640 1 -- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 msgr2=0x7f20340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2046d9d640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f20480c4690 0x7f20340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2046d9d640 1 -- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034003f30 msgr2=0x7f2034004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2046d9d640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034003f30 0x7f2034004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.612+0000 7f2046d9d640 1 -- 172.21.15.160:0/2043986407 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2034003520 con 0x7f2034004d40 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f2046d9d640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2034004d40 0x7f20340051c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2040009f90 tx=0x7f2034005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ba0a537d1101ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040009c10 con 0x7f2034004d40 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2040036070 con 0x7f2034004d40 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f2045d9b640 1 -- 172.21.15.160:0/2043986407 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2040031810 con 0x7f2034004d40 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f204cf61640 1 -- 172.21.15.160:0/2043986407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2034004d40 msgr2=0x7f20340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.619 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f204cf61640 1 --2- 172.21.15.160:0/2043986407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2034004d40 0x7f20340051c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2040009f90 tx=0x7f2034005830 comp rx=0 tx=0).stop 2024-05-09T07:14:08.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f2046d9d640 1 -- 172.21.15.160:0/2043986407 reap_dead start 2024-05-09T07:14:08.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f204cf61640 1 -- 172.21.15.160:0/2043986407 shutdown_connections 2024-05-09T07:14:08.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.613+0000 7f204cf61640 1 -- 172.21.15.160:0/2043986407 >> 172.21.15.160:0/2043986407 conn(0x7f2048072a80 msgr2=0x7f2048072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.614+0000 7f204cf61640 1 -- 172.21.15.160:0/2043986407 shutdown_connections 2024-05-09T07:14:08.620 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.614+0000 7f204cf61640 1 -- 172.21.15.160:0/2043986407 wait complete. 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.615+0000 7f204cf61640 1 Processor -- start 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.615+0000 7f204cf61640 1 -- start start 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034004d40 0x7f20480ad200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 0x7f204806c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f204806c7b0 0x7f2048151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f20480abc20 con 0x7f204809ac20 2024-05-09T07:14:08.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f20480c3af0 con 0x7f2034004d40 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2048070520 con 0x7f204806c7b0 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 0x7f204806c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 0x7f204806c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44378/0 (socket says 172.21.15.160:44378) 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 -- 172.21.15.160:0/621532218 learned_addr learned my addr 172.21.15.160:0/621532218 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f2046d9d640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f204806c7b0 0x7f2048151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204659c640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034004d40 0x7f20480ad200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 -- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f204806c7b0 msgr2=0x7f2048151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f204806c7b0 0x7f2048151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 -- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034004d40 msgr2=0x7f20480ad200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034004d40 0x7f20480ad200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 -- 172.21.15.160:0/621532218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204806cc30 con 0x7f204809ac20 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203dd9b640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 0x7f204806c270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20300017e0 tx=0x7f20300076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203000f9c0 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2030018070 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20481eb620 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.616+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f203000fde0 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.617+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20481eb8c0 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.617+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20480c5ca0 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.618+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f203001d460 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.618+0000 7f203f7fe640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f202806be60 0x7f202806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.619+0000 7f204659c640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f202806be60 0x7f202806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.619+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f203008b180 con 0x7f204809ac20 2024-05-09T07:14:08.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.619+0000 7f204659c640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f202806be60 0x7f202806e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2038009c40 tx=0x7f2038009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.623+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f204806cc30 con 0x7f204809ac20 2024-05-09T07:14:08.800 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.797+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f204809c870 con 0x7f204809ac20 2024-05-09T07:14:08.801 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.798+0000 7f203f7fe640 1 -- 172.21.15.160:0/621532218 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f2030059820 con 0x7f204809ac20 2024-05-09T07:14:08.801 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:14:08.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f202806be60 msgr2=0x7f202806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f202806be60 0x7f202806e320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2038009c40 tx=0x7f2038009190 comp rx=0 tx=0).stop 2024-05-09T07:14:08.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 msgr2=0x7f204806c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 0x7f204806c270 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f20300017e0 tx=0x7f20300076e0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.803 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 shutdown_connections 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f202806be60 0x7f202806e320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f204806c7b0 0x7f2048151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f204809ac20 0x7f204806c270 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 --2- 172.21.15.160:0/621532218 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f2034004d40 0x7f20480ad200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 >> 172.21.15.160:0/621532218 conn(0x7f2048072a80 msgr2=0x7f20480c6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 shutdown_connections 2024-05-09T07:14:08.804 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.800+0000 7f204cf61640 1 -- 172.21.15.160:0/621532218 wait complete. 2024-05-09T07:14:08.814 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:14:08.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.913+0000 7f0ab2ba9640 1 Processor -- start 2024-05-09T07:14:08.916 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.913+0000 7f0ab2ba9640 1 -- start start 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab2ba9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f0aac1550e0 0x7f0aac0c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab2ba9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac0ab310 0x7f0aac156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab2ba9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0aac06d510 0x7f0aac0c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab2ba9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f0aac06c500 con 0x7f0aac06d510 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab2ba9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f0aac06d080 con 0x7f0aac1550e0 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab2ba9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac0c20a0 con 0x7f0aac0ab310 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0aac06d510 0x7f0aac0c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f0aac06d510 0x7f0aac0c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44394/0 (socket says 172.21.15.160:44394) 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 -- 172.21.15.160:0/1929697229 learned_addr learned my addr 172.21.15.160:0/1929697229 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 -- 172.21.15.160:0/1929697229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac0ab310 msgr2=0x7f0aac156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab0945640 1 --2- 172.21.15.160:0/1929697229 >> v2:172.21.15.188:3300/0 conn(0x7f0aac1550e0 0x7f0aac0c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0aabfff640 1 --2- 172.21.15.160:0/1929697229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac0ab310 0x7f0aac156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 --2- 172.21.15.160:0/1929697229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac0ab310 0x7f0aac156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 -- 172.21.15.160:0/1929697229 >> v2:172.21.15.188:3300/0 conn(0x7f0aac1550e0 msgr2=0x7f0aac0c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 --2- 172.21.15.160:0/1929697229 >> v2:172.21.15.188:3300/0 conn(0x7f0aac1550e0 0x7f0aac0c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.914+0000 7f0ab1146640 1 -- 172.21.15.160:0/1929697229 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac06c750 con 0x7f0aac06d510 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.915+0000 7f0ab1146640 1 --2- 172.21.15.160:0/1929697229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac0c59c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0aac06c130 tx=0x7f0aa402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97dd846e41266c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.915+0000 7f0aab7fe640 1 -- 172.21.15.160:0/1929697229 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4036070 con 0x7f0aac06d510 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.915+0000 7f0aab7fe640 1 -- 172.21.15.160:0/1929697229 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0aa4031600 con 0x7f0aac06d510 2024-05-09T07:14:08.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.915+0000 7f0aab7fe640 1 -- 172.21.15.160:0/1929697229 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa40319d0 con 0x7f0aac06d510 2024-05-09T07:14:08.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/1929697229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 msgr2=0x7f0aac0c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/1929697229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac0c59c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0aac06c130 tx=0x7f0aa402a610 comp rx=0 tx=0).stop 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/1929697229 shutdown_connections 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/1929697229 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac0c59c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/1929697229 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac0ab310 0x7f0aac156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/1929697229 >> v2:172.21.15.188:3300/0 conn(0x7f0aac1550e0 0x7f0aac0c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/1929697229 >> 172.21.15.160:0/1929697229 conn(0x7f0aac0c69b0 msgr2=0x7f0aac0c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/1929697229 shutdown_connections 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.916+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/1929697229 wait complete. 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.917+0000 7f0ab2ba9640 1 Processor -- start 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.917+0000 7f0ab2ba9640 1 -- start start 2024-05-09T07:14:08.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac150b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aac0ab310 0x7f0aac1ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac1550e0 0x7f0aac1ea660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac1554c0 con 0x7f0aac06d510 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac06cd10 con 0x7f0aac0ab310 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac155640 con 0x7f0aac1550e0 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac150b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac150b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44402/0 (socket says 172.21.15.160:44402) 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 -- 172.21.15.160:0/3558294336 learned_addr learned my addr 172.21.15.160:0/3558294336 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab1146640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac1550e0 0x7f0aac1ea660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0aabfff640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aac0ab310 0x7f0aac1ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 -- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac1550e0 msgr2=0x7f0aac1ea660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac1550e0 0x7f0aac1ea660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.924 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 -- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aac0ab310 msgr2=0x7f0aac1ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aac0ab310 0x7f0aac1ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 -- 172.21.15.160:0/3558294336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac14d680 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab1146640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac1550e0 0x7f0aac1ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab0945640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac150b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c7f0 tx=0x7f0a9c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c015070 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0a9c004240 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aac0708a0 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.918+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c007620 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.919+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aac14d920 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.920+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aac14e030 con 0x7f0aac06d510 2024-05-09T07:14:08.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.920+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0a9c0079d0 con 0x7f0aac06d510 2024-05-09T07:14:08.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.920+0000 7f0aa97fa640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f0a9806be30 0x7f0a9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:08.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.921+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f0a9c08b190 con 0x7f0aac06d510 2024-05-09T07:14:08.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.921+0000 7f0aabfff640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f0a9806be30 0x7f0a9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:08.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.921+0000 7f0aabfff640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f0a9806be30 0x7f0a9806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0c1fc0 tx=0x7f0aa0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:08.927 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:08.925+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0a9c059890 con 0x7f0aac06d510 2024-05-09T07:14:09.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.095+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0aac075650 con 0x7f0aac06d510 2024-05-09T07:14:09.098 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.095+0000 7f0aa97fa640 1 -- 172.21.15.160:0/3558294336 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7f0a9c059190 con 0x7f0aac06d510 2024-05-09T07:14:09.099 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:14:09.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f0a9806be30 msgr2=0x7f0a9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f0a9806be30 0x7f0a9806e2f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0aac0c1fc0 tx=0x7f0aa0008040 comp rx=0 tx=0).stop 2024-05-09T07:14:09.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 msgr2=0x7f0aac150b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.100 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac150b30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c00c7f0 tx=0x7f0a9c00ccb0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 shutdown_connections 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f0a9806be30 0x7f0a9806e2f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0aac0ab310 0x7f0aac1ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0aac1550e0 0x7f0aac1ea660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 --2- 172.21.15.160:0/3558294336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0aac06d510 0x7f0aac150b30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 >> 172.21.15.160:0/3558294336 conn(0x7f0aac0c69b0 msgr2=0x7f0aac0c37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.097+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 shutdown_connections 2024-05-09T07:14:09.101 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.098+0000 7f0ab2ba9640 1 -- 172.21.15.160:0/3558294336 wait complete. 2024-05-09T07:14:09.111 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:14:09.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 Processor -- start 2024-05-09T07:14:09.212 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 -- start start 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf9c0c5d80 0x7fcf9c06b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf9c0a5e50 con 0x7fcf9c0c2f00 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcf9c153b20 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.209+0000 7fcfa284d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c1534d0 con 0x7fcf9c153fb0 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcfa0dea640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9b7fe640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf9c0c5d80 0x7fcf9c06b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9bfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fcf9c0c2f00 0x7fcf9c1568b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43384/0 (socket says 172.21.15.160:43384) 2024-05-09T07:14:09.213 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9bfff640 1 -- 172.21.15.160:0/62919593 learned_addr learned my addr 172.21.15.160:0/62919593 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 msgr2=0x7fcf9c15dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9b7fe640 1 --2- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c15dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/62919593 >> v1:172.21.15.188:6789/0 conn(0x7fcf9c0c2f00 legacy=0x7fcf9c1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/62919593 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c1537b0 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.210+0000 7fcf9affd640 1 -- 172.21.15.160:0/62919593 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3143653104 0 0) 0x7fcf9c0a5e50 con 0x7fcf9c0c2f00 2024-05-09T07:14:09.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.211+0000 7fcf9b7fe640 1 --2- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c06b160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf900097a0 tx=0x7fcf9002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5b4bbdb82636e1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.211+0000 7fcf9affd640 1 -- 172.21.15.160:0/62919593 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90037070 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.215 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.211+0000 7fcf9affd640 1 -- 172.21.15.160:0/62919593 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf9002adb0 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.211+0000 7fcf9affd640 1 -- 172.21.15.160:0/62919593 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90032660 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.211+0000 7fcfa284d640 1 -- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c5d80 msgr2=0x7fcf9c06b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.211+0000 7fcfa284d640 1 --2- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c06b160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcf900097a0 tx=0x7fcf9002a060 comp rx=0 tx=0).stop 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.212+0000 7fcfa284d640 1 -- 172.21.15.160:0/62919593 shutdown_connections 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.212+0000 7fcfa284d640 1 --2- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c06b160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.212+0000 7fcfa284d640 1 --2- 172.21.15.160:0/62919593 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c15dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.212+0000 7fcfa284d640 1 -- 172.21.15.160:0/62919593 >> 172.21.15.160:0/62919593 conn(0x7fcf9c0c7580 msgr2=0x7fcf9c0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.212+0000 7fcfa284d640 1 -- 172.21.15.160:0/62919593 shutdown_connections 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.212+0000 7fcfa284d640 1 -- 172.21.15.160:0/62919593 wait complete. 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 Processor -- start 2024-05-09T07:14:09.216 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 -- start start 2024-05-09T07:14:09.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c2f00 0x7fcf9c0726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c0c6730 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf9c0c65b0 con 0x7fcf9c0c2f00 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.213+0000 7fcfa284d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf9c0bb8f0 con 0x7fcf9c153fb0 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44414/0 (socket says 172.21.15.160:44414) 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/1685972525 learned_addr learned my addr 172.21.15.160:0/1685972525 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcfa0dea640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9bfff640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c2f00 0x7fcf9c0726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 msgr2=0x7fcf9c149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c2f00 msgr2=0x7fcf9c0726b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c2f00 0x7fcf9c0726b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 -- 172.21.15.160:0/1685972525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf9c070f30 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf9b7fe640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c149890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcf90035040 tx=0x7fcf900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90037040 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf90004030 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf90004420 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf9c151130 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.214+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9c0711d0 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.215+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf9c0756c0 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.220 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.216+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf90003610 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.216+0000 7fcf98ff9640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf8806be30 0x7fcf8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.216+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf900abd40 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.216+0000 7fcf9bfff640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf8806be30 0x7fcf8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.221 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.217+0000 7fcf9bfff640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf8806be30 0x7fcf8806e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c007fd0 tx=0x7fcf8c007de0 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.221+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf90079f10 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.431 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.429+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf9c0713a0 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.432 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.429+0000 7fcf98ff9640 1 -- 172.21.15.160:0/1685972525 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fcf9007e210 con 0x7fcf9c0c5d80 2024-05-09T07:14:09.432 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:09.432 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:14:00.864206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:14:09.432 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-05-09T07:14:09.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf8806be30 msgr2=0x7fcf8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf8806be30 0x7fcf8806e2f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c007fd0 tx=0x7fcf8c007de0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.434 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 msgr2=0x7fcf9c149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c149890 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcf90035040 tx=0x7fcf900046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 shutdown_connections 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf8806be30 0x7fcf8806e2f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf9c153fb0 0x7fcf9c149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf9c0c5d80 0x7fcf9c149890 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 --2- 172.21.15.160:0/1685972525 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf9c0c2f00 0x7fcf9c0726b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 >> 172.21.15.160:0/1685972525 conn(0x7fcf9c0c7580 msgr2=0x7fcf9c0c11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 shutdown_connections 2024-05-09T07:14:09.435 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.431+0000 7fcfa284d640 1 -- 172.21.15.160:0/1685972525 wait complete. 2024-05-09T07:14:09.445 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:14:09.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.545+0000 7f65fd35a640 1 Processor -- start 2024-05-09T07:14:09.548 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.545+0000 7f65fd35a640 1 -- start start 2024-05-09T07:14:09.549 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65fd35a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f80c24f0 0x7f65f80c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65fd35a640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f80c1d20 con 0x7f65f814f920 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65fd35a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65f80c5b90 con 0x7f65f80719e0 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65fd35a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f65f80c2270 con 0x7f65f80c24f0 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65f6ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f65f814f920 0x7f65f81561a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36456/0 (socket says 172.21.15.160:36456) 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65f6ffd640 1 -- 172.21.15.160:0/324312977 learned_addr learned my addr 172.21.15.160:0/324312977 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.546+0000 7f65f67fc640 1 --2- 172.21.15.160:0/324312977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f80c24f0 0x7f65f80c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.547+0000 7f65f67fc640 1 -- 172.21.15.160:0/324312977 >> v1:172.21.15.188:6789/0 conn(0x7f65f80719e0 legacy=0x7f65f80c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.547+0000 7f65f67fc640 1 -- 172.21.15.160:0/324312977 >> v1:172.21.15.160:6789/0 conn(0x7f65f814f920 legacy=0x7f65f81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.550 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.547+0000 7f65f67fc640 1 -- 172.21.15.160:0/324312977 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f80c1f90 con 0x7f65f80c24f0 2024-05-09T07:14:09.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.547+0000 7f65f67fc640 1 --2- 172.21.15.160:0/324312977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f80c24f0 0x7f65f80c3040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0097d0 tx=0x7f65ec02c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=245b7a1fd1f01432 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65f5ffb640 1 -- 172.21.15.160:0/324312977 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec038070 con 0x7f65f80c24f0 2024-05-09T07:14:09.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65f5ffb640 1 -- 172.21.15.160:0/324312977 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec031710 con 0x7f65f80c24f0 2024-05-09T07:14:09.551 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65f5ffb640 1 -- 172.21.15.160:0/324312977 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec031ae0 con 0x7f65f80c24f0 2024-05-09T07:14:09.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 -- 172.21.15.160:0/324312977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f80c24f0 msgr2=0x7f65f80c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 --2- 172.21.15.160:0/324312977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f80c24f0 0x7f65f80c3040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0097d0 tx=0x7f65ec02c420 comp rx=0 tx=0).stop 2024-05-09T07:14:09.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 -- 172.21.15.160:0/324312977 shutdown_connections 2024-05-09T07:14:09.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 --2- 172.21.15.160:0/324312977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f80c24f0 0x7f65f80c3040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.552 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 -- 172.21.15.160:0/324312977 >> 172.21.15.160:0/324312977 conn(0x7f65f80a98a0 msgr2=0x7f65f80a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:09.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 -- 172.21.15.160:0/324312977 shutdown_connections 2024-05-09T07:14:09.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.549+0000 7f65fd35a640 1 -- 172.21.15.160:0/324312977 wait complete. 2024-05-09T07:14:09.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 Processor -- start 2024-05-09T07:14:09.553 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 -- start start 2024-05-09T07:14:09.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f80719e0 0x7f65f8072cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 0x7f65f8149bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f814f920 0x7f65f8148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f65f80c3580 con 0x7f65f80c24f0 2024-05-09T07:14:09.555 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f65f814fd00 con 0x7f65f80719e0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65fd35a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f65f81511b0 con 0x7f65f814f920 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65f67fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 0x7f65f8149bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65f67fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 0x7f65f8149bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44420/0 (socket says 172.21.15.160:44420) 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.551+0000 7f65f67fc640 1 -- 172.21.15.160:0/3554614174 learned_addr learned my addr 172.21.15.160:0/3554614174 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f77fe640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f814f920 0x7f65f8148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f6ffd640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f80719e0 0x7f65f8072cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f67fc640 1 -- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f814f920 msgr2=0x7f65f8148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f67fc640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f814f920 0x7f65f8148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f67fc640 1 -- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f80719e0 msgr2=0x7f65f8072cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f67fc640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f80719e0 0x7f65f8072cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f67fc640 1 -- 172.21.15.160:0/3554614174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f81491f0 con 0x7f65f80c24f0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65f67fc640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 0x7f65f8149bc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65ec036040 tx=0x7f65ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec042020 con 0x7f65f80c24f0 2024-05-09T07:14:09.556 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f65ec033a40 con 0x7f65f80c24f0 2024-05-09T07:14:09.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec038040 con 0x7f65f80c24f0 2024-05-09T07:14:09.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f8148f50 con 0x7f65f80c24f0 2024-05-09T07:14:09.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.552+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f8073230 con 0x7f65f80c24f0 2024-05-09T07:14:09.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.553+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65f80756c0 con 0x7f65f80c24f0 2024-05-09T07:14:09.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.554+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f65ec045020 con 0x7f65f80c24f0 2024-05-09T07:14:09.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.554+0000 7f65df7fe640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f65e406be30 0x7f65e406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.554+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f65ec0adbb0 con 0x7f65f80c24f0 2024-05-09T07:14:09.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.554+0000 7f65f6ffd640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f65e406be30 0x7f65e406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.558 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.555+0000 7f65f6ffd640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f65e406be30 0x7f65e406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005880 tx=0x7f65e8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.558+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f65ec04c030 con 0x7f65f80c24f0 2024-05-09T07:14:09.733 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.730+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f65f80ab240 con 0x7f65f80c24f0 2024-05-09T07:14:09.734 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.732+0000 7f65df7fe640 1 -- 172.21.15.160:0/3554614174 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f65ec07bbd0 con 0x7f65f80c24f0 2024-05-09T07:14:09.735 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:09.735 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f65e406be30 msgr2=0x7f65e406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f65e406be30 0x7f65e406e2f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005880 tx=0x7f65e8009040 comp rx=0 tx=0).stop 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 msgr2=0x7f65f8149bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 0x7f65f8149bc0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f65ec036040 tx=0x7f65ec0046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 shutdown_connections 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f65e406be30 0x7f65e406e2f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f65f814f920 0x7f65f8148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65f80c24f0 0x7f65f8149bc0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 --2- 172.21.15.160:0/3554614174 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f65f80719e0 0x7f65f8072cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 >> 172.21.15.160:0/3554614174 conn(0x7f65f80a98a0 msgr2=0x7f65f80c4670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 shutdown_connections 2024-05-09T07:14:09.737 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.734+0000 7f65fd35a640 1 -- 172.21.15.160:0/3554614174 wait complete. 2024-05-09T07:14:09.748 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:14:09.748 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:14:09.851 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb652d4640 1 Processor -- start 2024-05-09T07:14:09.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb652d4640 1 -- start start 2024-05-09T07:14:09.852 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb652d4640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbb600c5690 0x7fbb600c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb652d4640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb60156d90 con 0x7fbb601568e0 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb652d4640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbb600c7ae0 con 0x7fbb60152a80 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb652d4640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fbb60152e60 con 0x7fbb600c5690 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb5f7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbb600c5690 0x7fbb600c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.849+0000 7fbb5e7fc640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fbb601568e0 0x7fbb60159310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36464/0 (socket says 172.21.15.160:36464) 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.850+0000 7fbb5e7fc640 1 -- 172.21.15.160:0/117547804 learned_addr learned my addr 172.21.15.160:0/117547804 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.850+0000 7fbb5f7fe640 1 -- 172.21.15.160:0/117547804 >> v1:172.21.15.188:6789/0 conn(0x7fbb60152a80 legacy=0x7fbb60155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.850+0000 7fbb5f7fe640 1 -- 172.21.15.160:0/117547804 >> v1:172.21.15.160:6789/0 conn(0x7fbb601568e0 legacy=0x7fbb60159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.853 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.850+0000 7fbb5f7fe640 1 -- 172.21.15.160:0/117547804 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb600c7dc0 con 0x7fbb600c5690 2024-05-09T07:14:09.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.850+0000 7fbb5f7fe640 1 --2- 172.21.15.160:0/117547804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600c5ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009d90 tx=0x7fbb5802a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7ef9a6547628a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb5dffb640 1 -- 172.21.15.160:0/117547804 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007740 con 0x7fbb600c5690 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb5dffb640 1 -- 172.21.15.160:0/117547804 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb580078e0 con 0x7fbb600c5690 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb5dffb640 1 -- 172.21.15.160:0/117547804 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58007cb0 con 0x7fbb600c5690 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 -- 172.21.15.160:0/117547804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 msgr2=0x7fbb600c5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 --2- 172.21.15.160:0/117547804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600c5ac0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009d90 tx=0x7fbb5802a580 comp rx=0 tx=0).stop 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 -- 172.21.15.160:0/117547804 shutdown_connections 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 --2- 172.21.15.160:0/117547804 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600c5ac0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 -- 172.21.15.160:0/117547804 >> 172.21.15.160:0/117547804 conn(0x7fbb60072ec0 msgr2=0x7fbb600ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 -- 172.21.15.160:0/117547804 shutdown_connections 2024-05-09T07:14:09.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.851+0000 7fbb652d4640 1 -- 172.21.15.160:0/117547804 wait complete. 2024-05-09T07:14:09.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.852+0000 7fbb652d4640 1 Processor -- start 2024-05-09T07:14:09.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.852+0000 7fbb652d4640 1 -- start start 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb652d4640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600cb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb652d4640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb60152a80 0x7fbb600c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb652d4640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb601568e0 0x7fbb600c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb652d4640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60158df0 con 0x7fbb600c5690 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb652d4640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbb600c80a0 con 0x7fbb60152a80 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb652d4640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbb600ad6a0 con 0x7fbb601568e0 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44426/0 (socket says 172.21.15.160:44426) 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 -- 172.21.15.160:0/3300063911 learned_addr learned my addr 172.21.15.160:0/3300063911 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5e7fc640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb60152a80 0x7fbb600c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5f7fe640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb601568e0 0x7fbb600c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 -- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb601568e0 msgr2=0x7fbb600c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb601568e0 0x7fbb600c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 -- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb60152a80 msgr2=0x7fbb600c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.853+0000 7fbb5effd640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb60152a80 0x7fbb600c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb5effd640 1 -- 172.21.15.160:0/3300063911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb601ec030 con 0x7fbb600c5690 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb5e7fc640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb60152a80 0x7fbb600c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb5f7fe640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb601568e0 0x7fbb600c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb5effd640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600cb700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b3f0 tx=0x7fbb5000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50015020 con 0x7fbb600c5690 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb50011830 con 0x7fbb600c5690 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb600c8c20 con 0x7fbb600c5690 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50011c20 con 0x7fbb600c5690 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.854+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb600ca0e0 con 0x7fbb600c5690 2024-05-09T07:14:09.859 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.855+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb601525c0 con 0x7fbb600c5690 2024-05-09T07:14:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.855+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbb50018020 con 0x7fbb600c5690 2024-05-09T07:14:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.856+0000 7fbb477fe640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbb4c06be30 0x7fbb4c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.856+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbb5008ad40 con 0x7fbb600c5690 2024-05-09T07:14:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.856+0000 7fbb5e7fc640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbb4c06be30 0x7fbb4c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:09.860 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.856+0000 7fbb5e7fc640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbb4c06be30 0x7fbb4c06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb60151e90 tx=0x7fbb54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:09.862 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:09.860+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbb5001f030 con 0x7fbb600c5690 2024-05-09T07:14:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.032+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbb60075650 con 0x7fbb600c5690 2024-05-09T07:14:10.035 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.032+0000 7fbb477fe640 1 -- 172.21.15.160:0/3300063911 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) ==== 93+0+10 (secure 0 0 0) 0x7fbb500172c0 con 0x7fbb600c5690 2024-05-09T07:14:10.035 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:14:10.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbb4c06be30 msgr2=0x7fbb4c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbb4c06be30 0x7fbb4c06e2f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbb60151e90 tx=0x7fbb54009040 comp rx=0 tx=0).stop 2024-05-09T07:14:10.037 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 msgr2=0x7fbb600cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600cb700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000b3f0 tx=0x7fbb5000b8b0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 shutdown_connections 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbb4c06be30 0x7fbb4c06e2f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbb60152a80 0x7fbb600c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbb601568e0 0x7fbb600c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 --2- 172.21.15.160:0/3300063911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb600c5690 0x7fbb600cb700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 >> 172.21.15.160:0/3300063911 conn(0x7fbb60072ec0 msgr2=0x7fbb600ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 shutdown_connections 2024-05-09T07:14:10.038 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.034+0000 7fbb652d4640 1 -- 172.21.15.160:0/3300063911 wait complete. 2024-05-09T07:14:10.048 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:14:10.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.148+0000 7fa9f0612640 1 Processor -- start 2024-05-09T07:14:10.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.148+0000 7fa9f0612640 1 -- start start 2024-05-09T07:14:10.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9f0612640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e8150800 0x7fa9e80a5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9f0612640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa9e80a9a60 0x7fa9e80c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9f0612640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa9e8153550 con 0x7fa9e80c6410 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9f0612640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa9e8153830 con 0x7fa9e80a9a60 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9f0612640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e8153b10 con 0x7fa9e8150800 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9eebaf640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa9e80c6410 0x7fa9e80c48b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36468/0 (socket says 172.21.15.160:36468) 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9eebaf640 1 -- 172.21.15.160:0/2030265341 learned_addr learned my addr 172.21.15.160:0/2030265341 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9ee3ae640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e8150800 0x7fa9e80a5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9edbad640 1 --2- 172.21.15.160:0/2030265341 >> v2:172.21.15.188:3300/0 conn(0x7fa9e80a9a60 0x7fa9e80c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9ed3ac640 1 -- 172.21.15.160:0/2030265341 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1799739455 0 0) 0x7fa9e8153550 con 0x7fa9e80c6410 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9ed3ac640 1 -- 172.21.15.160:0/2030265341 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa9d4003520 con 0x7fa9e80c6410 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9edbad640 1 -- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e8150800 msgr2=0x7fa9e80a5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9edbad640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e8150800 0x7fa9e80a5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9edbad640 1 -- 172.21.15.160:0/2030265341 >> v1:172.21.15.160:6789/0 conn(0x7fa9e80c6410 legacy=0x7fa9e80c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9edbad640 1 -- 172.21.15.160:0/2030265341 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e8153df0 con 0x7fa9e80a9a60 2024-05-09T07:14:10.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.149+0000 7fa9ee3ae640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e8150800 0x7fa9e80a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.150+0000 7fa9edbad640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e80a9a60 0x7fa9e80c5ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80097a0 tx=0x7fa9d802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28b669949cc22f7c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.150+0000 7fa9ed3ac640 1 -- 172.21.15.160:0/2030265341 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8037070 con 0x7fa9e80a9a60 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.150+0000 7fa9ed3ac640 1 -- 172.21.15.160:0/2030265341 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d802ae50 con 0x7fa9e80a9a60 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.150+0000 7fa9ed3ac640 1 -- 172.21.15.160:0/2030265341 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8032640 con 0x7fa9e80a9a60 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 -- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e80a9a60 msgr2=0x7fa9e80c5ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e80a9a60 0x7fa9e80c5ed0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80097a0 tx=0x7fa9d802a100 comp rx=0 tx=0).stop 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 -- 172.21.15.160:0/2030265341 shutdown_connections 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e80a9a60 0x7fa9e80c5ed0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 --2- 172.21.15.160:0/2030265341 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e8150800 0x7fa9e80a5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 -- 172.21.15.160:0/2030265341 >> 172.21.15.160:0/2030265341 conn(0x7fa9e80c69b0 msgr2=0x7fa9e80c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 -- 172.21.15.160:0/2030265341 shutdown_connections 2024-05-09T07:14:10.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.151+0000 7fa9f0612640 1 -- 172.21.15.160:0/2030265341 wait complete. 2024-05-09T07:14:10.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.152+0000 7fa9f0612640 1 Processor -- start 2024-05-09T07:14:10.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.152+0000 7fa9f0612640 1 -- start start 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9f0612640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e80a9a60 0x7fa9e80a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9f0612640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 0x7fa9e80731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9f0612640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e8150800 0x7fa9e8149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9f0612640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e80a6360 con 0x7fa9e80c6410 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9f0612640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8153200 con 0x7fa9e8150800 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9f0612640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa9e80c4fc0 con 0x7fa9e80a9a60 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 0x7fa9e80731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 0x7fa9e80731f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44430/0 (socket says 172.21.15.160:44430) 2024-05-09T07:14:10.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 -- 172.21.15.160:0/3851800054 learned_addr learned my addr 172.21.15.160:0/3851800054 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9ee3ae640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e80a9a60 0x7fa9e80a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9eebaf640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e8150800 0x7fa9e8149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 -- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e80a9a60 msgr2=0x7fa9e80a6e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e80a9a60 0x7fa9e80a6e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 -- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e8150800 msgr2=0x7fa9e8149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e8150800 0x7fa9e8149d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.153+0000 7fa9edbad640 1 -- 172.21.15.160:0/3851800054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e80bcab0 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.154+0000 7fa9edbad640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 0x7fa9e80731f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80097a0 tx=0x7fa9d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.154+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8042070 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.154+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa9d803e460 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.154+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e8072a30 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.154+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8037040 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.154+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e8070f70 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.155+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9e8075650 con 0x7fa9e80c6410 2024-05-09T07:14:10.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.155+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa9d8045070 con 0x7fa9e80c6410 2024-05-09T07:14:10.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.156+0000 7fa9deffd640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa9cc06be30 0x7fa9cc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.156+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa9d80abb40 con 0x7fa9e80c6410 2024-05-09T07:14:10.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.156+0000 7fa9ee3ae640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa9cc06be30 0x7fa9cc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.156+0000 7fa9ee3ae640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa9cc06be30 0x7fa9cc06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009a30 tx=0x7fa9e4008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.160+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa9d807a360 con 0x7fa9e80c6410 2024-05-09T07:14:10.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.334+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fa9e80a5f70 con 0x7fa9e80c6410 2024-05-09T07:14:10.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.335+0000 7fa9deffd640 1 -- 172.21.15.160:0/3851800054 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) ==== 104+0+11 (secure 0 0 0) 0x7fa9d8079c60 con 0x7fa9e80c6410 2024-05-09T07:14:10.338 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa9cc06be30 msgr2=0x7fa9cc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa9cc06be30 0x7fa9cc06e2f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4009a30 tx=0x7fa9e4008040 comp rx=0 tx=0).stop 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 msgr2=0x7fa9e80731f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 0x7fa9e80731f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80097a0 tx=0x7fa9d80046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 shutdown_connections 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa9cc06be30 0x7fa9cc06e2f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa9e8150800 0x7fa9e8149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa9e80c6410 0x7fa9e80731f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 --2- 172.21.15.160:0/3851800054 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa9e80a9a60 0x7fa9e80a6e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.340 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 >> 172.21.15.160:0/3851800054 conn(0x7fa9e80c69b0 msgr2=0x7fa9e80c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 shutdown_connections 2024-05-09T07:14:10.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.337+0000 7fa9f0612640 1 -- 172.21.15.160:0/3851800054 wait complete. 2024-05-09T07:14:10.351 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:14:10.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.451+0000 7f0574d29640 1 Processor -- start 2024-05-09T07:14:10.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0574d29640 1 -- start start 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0574d29640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f05701568e0 0x7f0570159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0574d29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05700c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0574d29640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0570156d90 con 0x7f0570152a80 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0574d29640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f0570152e60 con 0x7f05701568e0 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0574d29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f05700c80a0 con 0x7f05700c5690 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0567fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f05701568e0 0x7f0570159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.452+0000 7f0567fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f05701568e0 0x7f0570159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44438/0 (socket says 172.21.15.160:44438) 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f0567fff640 1 -- 172.21.15.160:0/2410424367 learned_addr learned my addr 172.21.15.160:0/2410424367 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:10.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f056ed9d640 1 --2- 172.21.15.160:0/2410424367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05700c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f0567fff640 1 -- 172.21.15.160:0/2410424367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 msgr2=0x7f05700c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:10.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f0567fff640 1 --2- 172.21.15.160:0/2410424367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05700c5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f0567fff640 1 -- 172.21.15.160:0/2410424367 >> v1:172.21.15.188:6789/0 conn(0x7f0570152a80 legacy=0x7f0570155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:10.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f0567fff640 1 -- 172.21.15.160:0/2410424367 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05700c7ae0 con 0x7f05701568e0 2024-05-09T07:14:10.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.453+0000 7f0567fff640 1 --2- 172.21.15.160:0/2410424367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f0570159310 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0570152fe0 tx=0x7f055802a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0b6f2ce1349390 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f056dd9b640 1 -- 172.21.15.160:0/2410424367 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558037070 con 0x7f05701568e0 2024-05-09T07:14:10.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f056dd9b640 1 -- 172.21.15.160:0/2410424367 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055802ade0 con 0x7f05701568e0 2024-05-09T07:14:10.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f056dd9b640 1 -- 172.21.15.160:0/2410424367 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0558032660 con 0x7f05701568e0 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 -- 172.21.15.160:0/2410424367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 msgr2=0x7f0570159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 --2- 172.21.15.160:0/2410424367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f0570159310 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0570152fe0 tx=0x7f055802a0d0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 -- 172.21.15.160:0/2410424367 shutdown_connections 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 --2- 172.21.15.160:0/2410424367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f0570159310 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 --2- 172.21.15.160:0/2410424367 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05700c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 -- 172.21.15.160:0/2410424367 >> 172.21.15.160:0/2410424367 conn(0x7f0570072ec0 msgr2=0x7f05700ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 -- 172.21.15.160:0/2410424367 shutdown_connections 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.454+0000 7f0574d29640 1 -- 172.21.15.160:0/2410424367 wait complete. 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.455+0000 7f0574d29640 1 Processor -- start 2024-05-09T07:14:10.459 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 -- start start 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05701ee500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570152a80 0x7f05701ec340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f05701ec880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f0570158df0 con 0x7f05701568e0 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0570158b10 con 0x7f0570152a80 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f0574d29640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f05700ad6a0 con 0x7f05700c5690 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f056ed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f05701ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f056ed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f05701ec880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44454/0 (socket says 172.21.15.160:44454) 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f056ed9d640 1 -- 172.21.15.160:0/4123516690 learned_addr learned my addr 172.21.15.160:0/4123516690 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:10.462 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.456+0000 7f056e59c640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05701ee500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0567fff640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570152a80 0x7f05701ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056ed9d640 1 -- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 msgr2=0x7f05701ee500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056ed9d640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05701ee500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056ed9d640 1 -- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570152a80 msgr2=0x7f05701ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056ed9d640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570152a80 0x7f05701ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056ed9d640 1 -- 172.21.15.160:0/4123516690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05700cb440 con 0x7f05701568e0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056e59c640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05701ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0567fff640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570152a80 0x7f05701ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f056ed9d640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f05701ec880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f056800cc30 tx=0x7f056800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f056800ce40 con 0x7f05701568e0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0568003990 con 0x7f05701568e0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05701eee40 con 0x7f05701568e0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0568003d40 con 0x7f05701568e0 2024-05-09T07:14:10.463 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.457+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05701ef120 con 0x7f05701568e0 2024-05-09T07:14:10.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.458+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0570075650 con 0x7f05701568e0 2024-05-09T07:14:10.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.458+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0568004000 con 0x7f05701568e0 2024-05-09T07:14:10.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.459+0000 7f0566ffd640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f055006be30 0x7f055006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.459+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f056808b5d0 con 0x7f05701568e0 2024-05-09T07:14:10.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.459+0000 7f056e59c640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f055006be30 0x7f055006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.464 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.459+0000 7f056e59c640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f055006be30 0x7f055006e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0560003eb0 tx=0x7f0560009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.463+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0568014030 con 0x7f05701568e0 2024-05-09T07:14:10.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.636+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f05700ad3a0 con 0x7f05701568e0 2024-05-09T07:14:10.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.636+0000 7f0566ffd640 1 -- 172.21.15.160:0/4123516690 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) ==== 100+0+11 (secure 0 0 0) 0x7f0568059cd0 con 0x7f05701568e0 2024-05-09T07:14:10.639 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f055006be30 msgr2=0x7f055006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f055006be30 0x7f055006e2f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0560003eb0 tx=0x7f0560009040 comp rx=0 tx=0).stop 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 msgr2=0x7f05701ec880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f05701ec880 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f056800cc30 tx=0x7f056800a5c0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 shutdown_connections 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f055006be30 0x7f055006e2f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.641 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f0570152a80 0x7f05701ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f05701568e0 0x7f05701ec880 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 --2- 172.21.15.160:0/4123516690 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f05700c5690 0x7f05701ee500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 >> 172.21.15.160:0/4123516690 conn(0x7f0570072ec0 msgr2=0x7f0570151410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 shutdown_connections 2024-05-09T07:14:10.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.638+0000 7f0574d29640 1 -- 172.21.15.160:0/4123516690 wait complete. 2024-05-09T07:14:10.652 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:14:10.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.752+0000 7fb5f31a5640 1 Processor -- start 2024-05-09T07:14:10.755 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.752+0000 7fb5f31a5640 1 -- start start 2024-05-09T07:14:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f31a5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb5ec0aa8e0 0x7fb5ec0c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.758 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f31a5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec155710 con 0x7fb5ec0c59c0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f31a5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5ec06dab0 con 0x7fb5ec0a6270 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f31a5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb5ec1559b0 con 0x7fb5ec0aa8e0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f0f41640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb5ec0c59c0 0x7fb5ec06d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36472/0 (socket says 172.21.15.160:36472) 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2911147316 learned_addr learned my addr 172.21.15.160:0/2911147316 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5f1742640 1 --2- 172.21.15.160:0/2911147316 >> v2:172.21.15.188:3300/0 conn(0x7fb5ec0aa8e0 0x7fb5ec0c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4254409575 0 0) 0x7fb5ec155710 con 0x7fb5ec0c59c0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb5d4003520 con 0x7fb5ec0c59c0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 77956819 0 0) 0x7fb5d4003520 con 0x7fb5ec0c59c0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 >> v2:172.21.15.188:3300/0 conn(0x7fb5ec0aa8e0 msgr2=0x7fb5ec0c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 --2- 172.21.15.160:0/2911147316 >> v2:172.21.15.188:3300/0 conn(0x7fb5ec0aa8e0 0x7fb5ec0c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 >> v1:172.21.15.188:6789/0 conn(0x7fb5ec0a6270 legacy=0x7fb5ec0aa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec155c90 con 0x7fb5ec0c59c0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7fb5e8001690 con 0x7fb5ec0c59c0 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 >> v1:172.21.15.160:6789/0 conn(0x7fb5ec0c59c0 legacy=0x7fb5ec06d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.753+0000 7fb5e37fe640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d4004800 0x7fb5d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5e37fe640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5d40051c0 0x7fb5d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec1559b0 con 0x7fb5d40039b0 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5d400c060 con 0x7fb5d4004800 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb5d400c260 con 0x7fb5d40051c0 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5e3fff640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d4004800 0x7fb5d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f1742640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5d40051c0 0x7fb5d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5d40051c0 msgr2=0x7fb5d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5d40051c0 0x7fb5d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d4004800 msgr2=0x7fb5d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5d4004800 0x7fb5d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2911147316 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5d4003520 con 0x7fb5d40039b0 2024-05-09T07:14:10.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f1742640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5d40051c0 0x7fb5d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.754+0000 7fb5f1742640 1 -- 172.21.15.160:0/2911147316 reap_dead start 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5d4003dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c210 tx=0x7fb5d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138cc86c314188ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007d10 con 0x7fb5d40039b0 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e8005ad0 con 0x7fb5d40039b0 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5e37fe640 1 -- 172.21.15.160:0/2911147316 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8007070 con 0x7fb5d40039b0 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2911147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 msgr2=0x7fb5d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5d4003dd0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c210 tx=0x7fb5d4005830 comp rx=0 tx=0).stop 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2911147316 shutdown_connections 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2911147316 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5d4003dd0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2911147316 >> 172.21.15.160:0/2911147316 conn(0x7fb5ec0c69b0 msgr2=0x7fb5ec0c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2911147316 shutdown_connections 2024-05-09T07:14:10.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.755+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2911147316 wait complete. 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.756+0000 7fb5f31a5640 1 Processor -- start 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.756+0000 7fb5f31a5640 1 -- start start 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f31a5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5ec1ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f31a5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5ec1de410 0x7fb5ec1e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f31a5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5ec0c0de0 0x7fb5ec148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f31a5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec14fba0 con 0x7fb5d40039b0 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f31a5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5ec14fa20 con 0x7fb5ec0c0de0 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f31a5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb5ec156490 con 0x7fb5ec1de410 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f0f41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5ec1ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f0f41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5ec1ea860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44474/0 (socket says 172.21.15.160:44474) 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.757+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2251782183 learned_addr learned my addr 172.21.15.160:0/2251782183 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2251782183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5ec1de410 msgr2=0x7fb5ec1e84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5ec1de410 0x7fb5ec1e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2251782183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5ec0c0de0 msgr2=0x7fb5ec148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5ec0c0de0 0x7fb5ec148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f0f41640 1 -- 172.21.15.160:0/2251782183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5ec1eb0a0 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f0f41640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5ec1ea860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c210 tx=0x7fb5ec155e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e803f020 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5e8001490 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80047e0 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5ec1dc3a0 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.758+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5ec1eada0 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.759+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5ec1dc1d0 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.759+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb5e8042020 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.760+0000 7fb5e17fa640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb5d806be30 0x7fb5d806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.760+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb5e80aac90 con 0x7fb5d40039b0 2024-05-09T07:14:10.765 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.760+0000 7fb5e3fff640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb5d806be30 0x7fb5d806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:10.766 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.760+0000 7fb5e3fff640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb5d806be30 0x7fb5d806e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:10.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.764+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5e8079300 con 0x7fb5d40039b0 2024-05-09T07:14:10.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.937+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb5ec075650 con 0x7fb5d40039b0 2024-05-09T07:14:10.940 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.938+0000 7fb5e17fa640 1 -- 172.21.15.160:0/2251782183 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) ==== 103+0+11 (secure 0 0 0) 0x7fb5e8078c00 con 0x7fb5d40039b0 2024-05-09T07:14:10.941 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb5d806be30 msgr2=0x7fb5d806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb5d806be30 0x7fb5d806e2f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc000c00 tx=0x7fb5dc009040 comp rx=0 tx=0).stop 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 msgr2=0x7fb5ec1ea860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5ec1ea860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb5e800c210 tx=0x7fb5ec155e60 comp rx=0 tx=0).stop 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 shutdown_connections 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb5d806be30 0x7fb5d806e2f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb5ec0c0de0 0x7fb5ec148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5ec1de410 0x7fb5ec1e84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 --2- 172.21.15.160:0/2251782183 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb5d40039b0 0x7fb5ec1ea860 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:10.943 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 >> 172.21.15.160:0/2251782183 conn(0x7fb5ec0c69b0 msgr2=0x7fb5ec14c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:10.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 shutdown_connections 2024-05-09T07:14:10.944 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:10.940+0000 7fb5f31a5640 1 -- 172.21.15.160:0/2251782183 wait complete. 2024-05-09T07:14:10.954 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:14:11.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.051+0000 7f926b1ce640 1 Processor -- start 2024-05-09T07:14:11.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.051+0000 7f926b1ce640 1 -- start start 2024-05-09T07:14:11.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926b1ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f92640c3ab0 0x7f92641532b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926b1ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9264072920 0x7f926406c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926b1ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9264157720 con 0x7f92640c6840 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926b1ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f92641568a0 con 0x7f9264072920 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926b1ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9264072f70 con 0x7f92640c3ab0 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f9263fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f92640c6840 0x7f9264157010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36476/0 (socket says 172.21.15.160:36476) 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f9263fff640 1 -- 172.21.15.160:0/4205167933 learned_addr learned my addr 172.21.15.160:0/4205167933 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926976b640 1 --2- 172.21.15.160:0/4205167933 >> v2:172.21.15.160:3300/0 conn(0x7f9264072920 0x7f926406c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f9268f6a640 1 --2- 172.21.15.160:0/4205167933 >> v2:172.21.15.188:3300/0 conn(0x7f92640c3ab0 0x7f92641532b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f92637fe640 1 -- 172.21.15.160:0/4205167933 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 18815265 0 0) 0x7f9264157720 con 0x7f92640c6840 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f92637fe640 1 -- 172.21.15.160:0/4205167933 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9254003520 con 0x7f92640c6840 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926976b640 1 -- 172.21.15.160:0/4205167933 >> v2:172.21.15.188:3300/0 conn(0x7f92640c3ab0 msgr2=0x7f92641532b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926976b640 1 --2- 172.21.15.160:0/4205167933 >> v2:172.21.15.188:3300/0 conn(0x7f92640c3ab0 0x7f92641532b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926976b640 1 -- 172.21.15.160:0/4205167933 >> v1:172.21.15.160:6789/0 conn(0x7f92640c6840 legacy=0x7f9264157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.052+0000 7f926976b640 1 -- 172.21.15.160:0/4205167933 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9264073250 con 0x7f9264072920 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.053+0000 7f926976b640 1 --2- 172.21.15.160:0/4205167933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9264072920 0x7f926406c6d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9264156a20 tx=0x7f925c02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f79d7a87295ff55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.053+0000 7f92637fe640 1 -- 172.21.15.160:0/4205167933 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c031600 con 0x7f9264072920 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.053+0000 7f92637fe640 1 -- 172.21.15.160:0/4205167933 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925c0317a0 con 0x7f9264072920 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.053+0000 7f92637fe640 1 -- 172.21.15.160:0/4205167933 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c031b70 con 0x7f9264072920 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 -- 172.21.15.160:0/4205167933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9264072920 msgr2=0x7f926406c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 --2- 172.21.15.160:0/4205167933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9264072920 0x7f926406c6d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9264156a20 tx=0x7f925c02a5d0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 -- 172.21.15.160:0/4205167933 shutdown_connections 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 --2- 172.21.15.160:0/4205167933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9264072920 0x7f926406c6d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 --2- 172.21.15.160:0/4205167933 >> v2:172.21.15.188:3300/0 conn(0x7f92640c3ab0 0x7f92641532b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 -- 172.21.15.160:0/4205167933 >> 172.21.15.160:0/4205167933 conn(0x7f92640a9c90 msgr2=0x7f92640aa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 -- 172.21.15.160:0/4205167933 shutdown_connections 2024-05-09T07:14:11.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.054+0000 7f926b1ce640 1 -- 172.21.15.160:0/4205167933 wait complete. 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.055+0000 7f926b1ce640 1 Processor -- start 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.055+0000 7f926b1ce640 1 -- start start 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9264072920 0x7f92640bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92640c3ab0 0x7f92641e8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 0x7f92640ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f92640c3e90 con 0x7f92640c6840 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f92640c5a30 con 0x7f92640c3ab0 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f92640c4010 con 0x7f9264072920 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 0x7f92640ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 0x7f92640ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44490/0 (socket says 172.21.15.160:44490) 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 -- 172.21.15.160:0/3560983806 learned_addr learned my addr 172.21.15.160:0/3560983806 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f9268f6a640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9264072920 0x7f92640bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f9263fff640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92640c3ab0 0x7f92641e8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 -- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9264072920 msgr2=0x7f92640bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9264072920 0x7f92640bcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 -- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92640c3ab0 msgr2=0x7f92641e8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92640c3ab0 0x7f92641e8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 -- 172.21.15.160:0/3560983806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92640ac3e0 con 0x7f92640c6840 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926976b640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 0x7f92640ad400 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f925c02abc0 tx=0x7f925c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c03f070 con 0x7f92640c6840 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f925c03b820 con 0x7f92640c6840 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.056+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92640ac6c0 con 0x7f92640c6840 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.057+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c03bbd0 con 0x7f92640c6840 2024-05-09T07:14:11.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.057+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92640acee0 con 0x7f92640c6840 2024-05-09T07:14:11.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.058+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92640756c0 con 0x7f92640c6840 2024-05-09T07:14:11.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.058+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f925c04d3f0 con 0x7f92640c6840 2024-05-09T07:14:11.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.058+0000 7f92617fa640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f924806be30 0x7f924806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.059+0000 7f9268f6a640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f924806be30 0x7f924806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.059+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f925c0ac430 con 0x7f92640c6840 2024-05-09T07:14:11.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.059+0000 7f9268f6a640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f924806be30 0x7f924806e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92640a89d0 tx=0x7f9258009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.063+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f925c07a4e0 con 0x7f92640c6840 2024-05-09T07:14:11.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.274+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f92640c6620 con 0x7f92640c6840 2024-05-09T07:14:11.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.275+0000 7f92617fa640 1 -- 172.21.15.160:0/3560983806 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f925c07e7e0 con 0x7f92640c6840 2024-05-09T07:14:11.278 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:11.278 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:14:00.864206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:14:11.278 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f924806be30 msgr2=0x7f924806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f924806be30 0x7f924806e2f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f92640a89d0 tx=0x7f9258009190 comp rx=0 tx=0).stop 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 msgr2=0x7f92640ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 0x7f92640ad400 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f925c02abc0 tx=0x7f925c004a40 comp rx=0 tx=0).stop 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 shutdown_connections 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f924806be30 0x7f924806e2f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f92640c3ab0 0x7f92641e8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f92640c6840 0x7f92640ad400 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 --2- 172.21.15.160:0/3560983806 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9264072920 0x7f92640bcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 >> 172.21.15.160:0/3560983806 conn(0x7f92640a9c90 msgr2=0x7f92640a8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:11.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 shutdown_connections 2024-05-09T07:14:11.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.277+0000 7f926b1ce640 1 -- 172.21.15.160:0/3560983806 wait complete. 2024-05-09T07:14:11.291 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:14:11.391 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.389+0000 7f71a7fd5640 1 Processor -- start 2024-05-09T07:14:11.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.389+0000 7f71a7fd5640 1 -- start start 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a7fd5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f71a00be320 0x7f71a00c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a7fd5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71a00c8b90 0x7f71a0155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a7fd5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71a0159840 con 0x7f71a00c5610 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a7fd5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f71a0159310 con 0x7f71a00be320 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a7fd5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f71a0159560 con 0x7f71a00c8b90 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5d71640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f71a00c5610 0x7f71a0155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36484/0 (socket says 172.21.15.160:36484) 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5570640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f71a00be320 0x7f71a00c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.393 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5d71640 1 -- 172.21.15.160:0/2872104886 learned_addr learned my addr 172.21.15.160:0/2872104886 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:11.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a6572640 1 --2- 172.21.15.160:0/2872104886 >> v2:172.21.15.188:3300/0 conn(0x7f71a00c8b90 0x7f71a0155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5570640 1 -- 172.21.15.160:0/2872104886 >> v2:172.21.15.188:3300/0 conn(0x7f71a00c8b90 msgr2=0x7f71a0155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:11.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5570640 1 --2- 172.21.15.160:0/2872104886 >> v2:172.21.15.188:3300/0 conn(0x7f71a00c8b90 0x7f71a0155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5570640 1 -- 172.21.15.160:0/2872104886 >> v1:172.21.15.160:6789/0 conn(0x7f71a00c5610 legacy=0x7f71a0155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a5570640 1 -- 172.21.15.160:0/2872104886 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a00c77d0 con 0x7f71a00be320 2024-05-09T07:14:11.394 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.390+0000 7f71a4d6f640 1 -- 172.21.15.160:0/2872104886 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 609448167 0 0) 0x7f71a0159840 con 0x7f71a00c5610 2024-05-09T07:14:11.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a5570640 1 --2- 172.21.15.160:0/2872104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00be320 0x7f71a00c59f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a00ad050 tx=0x7f719002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4925cc118024dc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a4d6f640 1 -- 172.21.15.160:0/2872104886 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190037070 con 0x7f71a00be320 2024-05-09T07:14:11.395 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a4d6f640 1 -- 172.21.15.160:0/2872104886 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f719002add0 con 0x7f71a00be320 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a4d6f640 1 -- 172.21.15.160:0/2872104886 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7190032660 con 0x7f71a00be320 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a7fd5640 1 -- 172.21.15.160:0/2872104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00be320 msgr2=0x7f71a00c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/2872104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00be320 0x7f71a00c59f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f71a00ad050 tx=0x7f719002a080 comp rx=0 tx=0).stop 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a7fd5640 1 -- 172.21.15.160:0/2872104886 shutdown_connections 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/2872104886 >> v2:172.21.15.188:3300/0 conn(0x7f71a00c8b90 0x7f71a0155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/2872104886 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00be320 0x7f71a00c59f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.391+0000 7f71a7fd5640 1 -- 172.21.15.160:0/2872104886 >> 172.21.15.160:0/2872104886 conn(0x7f71a0072e90 msgr2=0x7f71a00ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.392+0000 7f71a7fd5640 1 -- 172.21.15.160:0/2872104886 shutdown_connections 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.392+0000 7f71a7fd5640 1 -- 172.21.15.160:0/2872104886 wait complete. 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 Processor -- start 2024-05-09T07:14:11.396 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 -- start start 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a00be320 0x7f71a0157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a00c5610 0x7f71a014be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 0x7f71a014c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f71a00ad6a0 con 0x7f71a00c8b90 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71a00c7d90 con 0x7f71a00c5610 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a7fd5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71a0155f60 con 0x7f71a00be320 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a6572640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 0x7f71a014c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a6572640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 0x7f71a014c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44512/0 (socket says 172.21.15.160:44512) 2024-05-09T07:14:11.399 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.393+0000 7f71a6572640 1 -- 172.21.15.160:0/3070642518 learned_addr learned my addr 172.21.15.160:0/3070642518 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a5570640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a00c5610 0x7f71a014be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a5d71640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a00be320 0x7f71a0157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a6572640 1 -- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a00be320 msgr2=0x7f71a0157db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a6572640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a00be320 0x7f71a0157db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a6572640 1 -- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a00c5610 msgr2=0x7f71a014be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a6572640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a00c5610 0x7f71a014be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a6572640 1 -- 172.21.15.160:0/3070642518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a01ec150 con 0x7f71a00c8b90 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a5570640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a00c5610 0x7f71a014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a6572640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 0x7f71a014c380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f719800efc0 tx=0x7f719800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198018020 con 0x7f71a00c8b90 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7198014480 con 0x7f71a00c8b90 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a0153810 con 0x7f71a00c8b90 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198014870 con 0x7f71a00c8b90 2024-05-09T07:14:11.400 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.394+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a01ec410 con 0x7f71a00c8b90 2024-05-09T07:14:11.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.395+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71a00756c0 con 0x7f71a00c8b90 2024-05-09T07:14:11.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.396+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f7198005d10 con 0x7f71a00c8b90 2024-05-09T07:14:11.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.396+0000 7f71967fc640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f718406be30 0x7f718406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.396+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f719808bf20 con 0x7f71a00c8b90 2024-05-09T07:14:11.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.396+0000 7f71a5d71640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f718406be30 0x7f718406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.397+0000 7f71a5d71640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f718406be30 0x7f718406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f60 tx=0x7f719c007b50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.403 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.400+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7198022030 con 0x7f71a00c8b90 2024-05-09T07:14:11.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.578+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71a00ad340 con 0x7f71a00c8b90 2024-05-09T07:14:11.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.579+0000 7f71967fc640 1 -- 172.21.15.160:0/3070642518 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18550 (secure 0 0 0) 0x7f7198010040 con 0x7f71a00c8b90 2024-05-09T07:14:11.582 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:11.582 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":25,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:13:51.486600+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:14:11.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f718406be30 msgr2=0x7f718406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f718406be30 0x7f718406e2f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f719c000f60 tx=0x7f719c007b50 comp rx=0 tx=0).stop 2024-05-09T07:14:11.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 msgr2=0x7f71a014c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 0x7f71a014c380 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f719800efc0 tx=0x7f719800c490 comp rx=0 tx=0).stop 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 shutdown_connections 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f718406be30 0x7f718406e2f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71a00c5610 0x7f71a014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71a00c8b90 0x7f71a014c380 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 --2- 172.21.15.160:0/3070642518 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71a00be320 0x7f71a0157db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 >> 172.21.15.160:0/3070642518 conn(0x7f71a0072e90 msgr2=0x7f71a014e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 shutdown_connections 2024-05-09T07:14:11.585 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.581+0000 7f71a7fd5640 1 -- 172.21.15.160:0/3070642518 wait complete. 2024-05-09T07:14:11.596 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:14:11.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.693+0000 7fcf52ecd640 1 Processor -- start 2024-05-09T07:14:11.696 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.693+0000 7fcf52ecd640 1 -- start start 2024-05-09T07:14:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf52ecd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf52ecd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf4c153fb0 0x7fcf4c15dc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.697 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf52ecd640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcf4c0a5e50 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf52ecd640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fcf4c153b20 con 0x7fcf4c153fb0 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf52ecd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c1534d0 con 0x7fcf4c0c2f00 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf5146a640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fcf4c153fb0 0x7fcf4c15dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf50c69640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf4bfff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fcf4c0c5d80 0x7fcf4c06b160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:43450/0 (socket says 172.21.15.160:43450) 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.694+0000 7fcf4bfff640 1 -- 172.21.15.160:0/1341315717 learned_addr learned my addr 172.21.15.160:0/1341315717 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.695+0000 7fcf5146a640 1 -- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 msgr2=0x7fcf4c1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.695+0000 7fcf5146a640 1 --2- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c1568b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.695+0000 7fcf5146a640 1 -- 172.21.15.160:0/1341315717 >> v1:172.21.15.188:6789/0 conn(0x7fcf4c0c5d80 legacy=0x7fcf4c06b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.695+0000 7fcf5146a640 1 -- 172.21.15.160:0/1341315717 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1537b0 con 0x7fcf4c153fb0 2024-05-09T07:14:11.698 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.695+0000 7fcf50c69640 1 --2- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.695+0000 7fcf5146a640 1 --2- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c15dc40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009d90 tx=0x7fcf4402a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905131c2c3f18af2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf4b7fe640 1 -- 172.21.15.160:0/1341315717 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44007740 con 0x7fcf4c153fb0 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf4b7fe640 1 -- 172.21.15.160:0/1341315717 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf440078e0 con 0x7fcf4c153fb0 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf4b7fe640 1 -- 172.21.15.160:0/1341315717 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44007cb0 con 0x7fcf4c153fb0 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 -- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 msgr2=0x7fcf4c15dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c15dc40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009d90 tx=0x7fcf4402a530 comp rx=0 tx=0).stop 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 -- 172.21.15.160:0/1341315717 shutdown_connections 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c15dc40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.700 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/1341315717 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 -- 172.21.15.160:0/1341315717 >> 172.21.15.160:0/1341315717 conn(0x7fcf4c0c7580 msgr2=0x7fcf4c0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:11.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 -- 172.21.15.160:0/1341315717 shutdown_connections 2024-05-09T07:14:11.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.696+0000 7fcf52ecd640 1 -- 172.21.15.160:0/1341315717 wait complete. 2024-05-09T07:14:11.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.697+0000 7fcf52ecd640 1 Processor -- start 2024-05-09T07:14:11.701 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.697+0000 7fcf52ecd640 1 -- start start 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf52ecd640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf52ecd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 0x7fcf4c142de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf52ecd640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c143320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf52ecd640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0c6730 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf52ecd640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fcf4c0c65b0 con 0x7fcf4c153fb0 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf52ecd640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fcf4c0bb8f0 con 0x7fcf4c0c2f00 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 0x7fcf4c142de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf4bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 0x7fcf4c142de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44520/0 (socket says 172.21.15.160:44520) 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf4bfff640 1 -- 172.21.15.160:0/375739150 learned_addr learned my addr 172.21.15.160:0/375739150 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf50c69640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.698+0000 7fcf5146a640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c143320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf4bfff640 1 -- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 msgr2=0x7fcf4c073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.703 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf4bfff640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf4bfff640 1 -- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 msgr2=0x7fcf4c143320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf4bfff640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c143320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf4bfff640 1 -- 172.21.15.160:0/375739150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf4c1e9dc0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf50c69640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf4bfff640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 0x7fcf4c142de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c470 tx=0x7fcf4000c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf40017070 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcf40005ce0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf400072d0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf4c1ea050 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.699+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf4c1dcfd0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.700+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf4c1515f0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.700+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fcf4001a070 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.701+0000 7fcf497fa640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf3806be30 0x7fcf3806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:11.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.701+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcf4008a300 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.701+0000 7fcf50c69640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf3806be30 0x7fcf3806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:11.705 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.701+0000 7fcf50c69640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf3806be30 0x7fcf3806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009a20 tx=0x7fcf3c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:11.707 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.705+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fcf4c1515f0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.918 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.915+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf4c0756c0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.919 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.916+0000 7fcf497fa640 1 -- 172.21.15.160:0/375739150 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fcf400583b0 con 0x7fcf4c0c5d80 2024-05-09T07:14:11.919 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:14:11.919 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:14:00.864206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:14:11.919 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf3806be30 msgr2=0x7fcf3806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf3806be30 0x7fcf3806e2f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c009a20 tx=0x7fcf3c008040 comp rx=0 tx=0).stop 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 msgr2=0x7fcf4c142de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 0x7fcf4c142de0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcf4000c470 tx=0x7fcf4000c930 comp rx=0 tx=0).stop 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 shutdown_connections 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fcf3806be30 0x7fcf3806e2f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fcf4c153fb0 0x7fcf4c143320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf4c0c5d80 0x7fcf4c142de0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 --2- 172.21.15.160:0/375739150 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fcf4c0c2f00 0x7fcf4c073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:11.921 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 >> 172.21.15.160:0/375739150 conn(0x7fcf4c0c7580 msgr2=0x7fcf4c150ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:11.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 shutdown_connections 2024-05-09T07:14:11.922 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:11.918+0000 7fcf52ecd640 1 -- 172.21.15.160:0/375739150 wait complete. 2024-05-09T07:14:12.334 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-09T07:13:44.804208+0000', 'modified': '2024-05-09T07:14:00.864206+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.188:6833/3545719682', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.188:6832', 'nonce': 3545719682}, {'type': 'v1', 'addr': '172.21.15.188:6833', 'nonce': 3545719682}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4296, 'qdb_cluster': [4296]} 2024-05-09T07:14:12.335 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-09T07:14:12.335 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-05-09T07:14:12.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.434+0000 7fa4136e7640 1 Processor -- start 2024-05-09T07:14:12.437 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.434+0000 7fa4136e7640 1 -- start start 2024-05-09T07:14:12.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa4136e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa40c153300 0x7fa40c0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa4136e7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40c1563b0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.438 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa4136e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa40c156690 con 0x7fa40c0a5fb0 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa4136e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa40c0c1630 con 0x7fa40c153300 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa411c84640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa40c0c5e20 0x7fa40c15cc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:36498/0 (socket says 172.21.15.160:36498) 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa411c84640 1 -- 172.21.15.160:0/766757787 learned_addr learned my addr 172.21.15.160:0/766757787 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa411483640 1 --2- 172.21.15.160:0/766757787 >> v2:172.21.15.188:3300/0 conn(0x7fa40c153300 0x7fa40c0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1290834072 0 0) 0x7fa40c1563b0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3f8003520 con 0x7fa40c0c5e20 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4040574841 0 0) 0x7fa40c156690 con 0x7fa40c0a5fb0 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa40c1563b0 con 0x7fa40c0a5fb0 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 641978614 0 0) 0x7fa3f8003520 con 0x7fa40c0c5e20 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa411483640 1 -- 172.21.15.160:0/766757787 >> v1:172.21.15.188:6789/0 conn(0x7fa40c0a5fb0 legacy=0x7fa40c0bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa411483640 1 -- 172.21.15.160:0/766757787 >> v1:172.21.15.160:6789/0 conn(0x7fa40c0c5e20 legacy=0x7fa40c15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.439 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.435+0000 7fa411483640 1 -- 172.21.15.160:0/766757787 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c0c1c20 con 0x7fa40c153300 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.436+0000 7fa411483640 1 --2- 172.21.15.160:0/766757787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c0c58e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4080097a0 tx=0x7fa40802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a800811d5239e2e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.436+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408037070 con 0x7fa40c153300 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.436+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40802ac50 con 0x7fa40c153300 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.436+0000 7fa403fff640 1 -- 172.21.15.160:0/766757787 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040019f0 con 0x7fa40c153300 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 -- 172.21.15.160:0/766757787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 msgr2=0x7fa40c0c58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 --2- 172.21.15.160:0/766757787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c0c58e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa4080097a0 tx=0x7fa40802a1c0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 -- 172.21.15.160:0/766757787 shutdown_connections 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 --2- 172.21.15.160:0/766757787 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c0c58e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 -- 172.21.15.160:0/766757787 >> 172.21.15.160:0/766757787 conn(0x7fa40c0c7580 msgr2=0x7fa40c0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 -- 172.21.15.160:0/766757787 shutdown_connections 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.437+0000 7fa4136e7640 1 -- 172.21.15.160:0/766757787 wait complete. 2024-05-09T07:14:12.441 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.438+0000 7fa4136e7640 1 Processor -- start 2024-05-09T07:14:12.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.438+0000 7fa4136e7640 1 -- start start 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c0a5fb0 0x7fa40c1e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 0x7fa40c0c01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c1ecd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0c2420 con 0x7fa40c0c5e20 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c0c19d0 con 0x7fa40c153300 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c0bd220 con 0x7fa40c0a5fb0 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 0x7fa40c0c01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 0x7fa40c0c01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44530/0 (socket says 172.21.15.160:44530) 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 -- 172.21.15.160:0/4013252826 learned_addr learned my addr 172.21.15.160:0/4013252826 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa411483640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c0a5fb0 0x7fa40c1e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa411c84640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c1ecd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 -- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c0a5fb0 msgr2=0x7fa40c1e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c0a5fb0 0x7fa40c1e8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 -- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 msgr2=0x7fa40c1ecd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c1ecd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 -- 172.21.15.160:0/4013252826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40c0abed0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa411483640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c0a5fb0 0x7fa40c1e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa411c84640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c1ecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa410c82640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 0x7fa40c0c01a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00c430 tx=0x7fa3fc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa401ffb640 1 -- 172.21.15.160:0/4013252826 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc017070 con 0x7fa40c0c5e20 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa401ffb640 1 -- 172.21.15.160:0/4013252826 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3fc0052f0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40c0bdde0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.439+0000 7fa401ffb640 1 -- 172.21.15.160:0/4013252826 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc003830 con 0x7fa40c0c5e20 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.440+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa40c0c06e0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.441+0000 7fa401ffb640 1 -- 172.21.15.160:0/4013252826 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa3fc01a070 con 0x7fa40c0c5e20 2024-05-09T07:14:12.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.442+0000 7fa401ffb640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa3f006be30 0x7fa3f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.442+0000 7fa401ffb640 1 -- 172.21.15.160:0/4013252826 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3fc08a5f0 con 0x7fa40c0c5e20 2024-05-09T07:14:12.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.442+0000 7fa411483640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa3f006be30 0x7fa3f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.448 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.442+0000 7fa411483640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa3f006be30 0x7fa3f006e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa408034000 tx=0x7fa40802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 Processor -- start 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 -- start start 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6292f0 0x55ca0f62b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f62a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 0x55ca0f5e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f5504d0 con 0x55ca0f629c30 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f4f83e0 con 0x55ca0f6292f0 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55ca0f54c490 con 0x55ca0f6296d0 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa411c84640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 0x55ca0f5e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa411c84640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 0x55ca0f5e7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44546/0 (socket says 172.21.15.160:44546) 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.443+0000 7fa411c84640 1 -- 172.21.15.160:0/2203696851 learned_addr learned my addr 172.21.15.160:0/2203696851 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa410c82640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f62a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411483640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6292f0 0x55ca0f62b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.449 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411c84640 1 -- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 msgr2=0x55ca0f62a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411c84640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f62a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411c84640 1 -- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6292f0 msgr2=0x55ca0f62b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411c84640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6292f0 0x55ca0f62b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411c84640 1 -- 172.21.15.160:0/2203696851 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0f2642d0 con 0x55ca0f629c30 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa410c82640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f62a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411483640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6292f0 0x55ca0f62b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa411c84640 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 0x55ca0f5e7a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa404001780 tx=0x7fa40400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38649449c5200a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa3f6ffd640 1 -- 172.21.15.160:0/2203696851 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404019070 con 0x55ca0f629c30 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa3f6ffd640 1 -- 172.21.15.160:0/2203696851 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40400ee40 con 0x55ca0f629c30 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.444+0000 7fa3f6ffd640 1 -- 172.21.15.160:0/2203696851 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4040074e0 con 0x55ca0f629c30 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 -- 172.21.15.160:0/2203696851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 msgr2=0x55ca0f5e7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 0x55ca0f5e7a10 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa404001780 tx=0x7fa40400e670 comp rx=0 tx=0).stop 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 -- 172.21.15.160:0/2203696851 shutdown_connections 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f629c30 0x55ca0f5e7a10 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.450 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f62a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2203696851 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6292f0 0x55ca0f62b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 -- 172.21.15.160:0/2203696851 >> 172.21.15.160:0/2203696851 conn(0x55ca0f5e9b60 msgr2=0x55ca0f60e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 -- 172.21.15.160:0/2203696851 shutdown_connections 2024-05-09T07:14:12.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 -- 172.21.15.160:0/2203696851 wait complete. 2024-05-09T07:14:12.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 -- start start 2024-05-09T07:14:12.451 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.445+0000 7fa4163f0740 1 Processor -- start 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f6082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f629c30 0x55ca0f626160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 0x55ca0f641b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f5e8de0 con 0x55ca0f641710 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f50b8e0 con 0x55ca0f629c30 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55ca0f636ac0 con 0x55ca0f6296d0 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa4163f0740 10 client.0 _ll_register_callbacks cb 0x55ca0f4d89c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 0x55ca0f641b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 0x55ca0f641b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44558/0 (socket says 172.21.15.160:44558) 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 -- 172.21.15.160:0/1211141933 learned_addr learned my addr 172.21.15.160:0/1211141933 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411483640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f6082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa410c82640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f629c30 0x55ca0f626160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 -- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 msgr2=0x55ca0f6082a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f6082a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 -- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f629c30 msgr2=0x55ca0f626160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f629c30 0x55ca0f626160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.446+0000 7fa411c84640 1 -- 172.21.15.160:0/1211141933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0f32e6b0 con 0x55ca0f641710 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa411483640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f6082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa411c84640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 0x55ca0f641b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa404005c00 tx=0x7fa4040014d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6852facd9e555b28 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa3ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa3ebfff640 1 -- 172.21.15.160:0/1211141933 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa404019040 con 0x55ca0f641710 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa3ebfff640 1 -- 172.21.15.160:0/1211141933 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4040078d0 con 0x55ca0f641710 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ca0f55dc00 con 0x55ca0f641710 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa3ebfff640 1 -- 172.21.15.160:0/1211141933 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa40402b050 con 0x55ca0f641710 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 10 client.4860 fetch_fsmap learned FSMap version 8 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ca0f625750 con 0x55ca0f641710 2024-05-09T07:14:12.456 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa3ebfff640 1 -- 172.21.15.160:0/1211141933 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x55ca0f55dc00 con 0x55ca0f641710 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa3ebfff640 1 -- 172.21.15.160:0/1211141933 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fa40402e020 con 0x55ca0f641710 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 10 client.4860 fetch_fsmap finished waiting for FSMap version 8 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 10 client.4860 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 20 client.4860 populate_metadata read hostname 'smithi160' 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f623cb0 0x55ca0f6078a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 4 client.4860 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 --> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca0f557f50 con 0x55ca0f623cb0 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.447+0000 7fa411483640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f623cb0 0x55ca0f6078a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.448+0000 7fa3ebfff640 10 client.4860 ms_handle_connect on v2:172.21.15.188:6832/3545719682 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.448+0000 7fa411483640 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f623cb0 0x55ca0f6078a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb81563a02ad79e server_cookie=81bd62baea464a08 in_seq=0 out_seq=0 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.448+0000 7fa3ebfff640 1 -- 172.21.15.160:0/1211141933 <== mds.0 v2:172.21.15.188:6832/3545719682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ca0f557f50 con 0x55ca0f623cb0 2024-05-09T07:14:12.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.448+0000 7fa3ebfff640 10 client.4860 handle_command_reply: tid=0 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 client.4860 shutdown 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 msgr2=0x55ca0f641b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 0x55ca0f641b90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa404005c00 tx=0x7fa4040014d0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 shutdown_connections 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f623cb0 0x55ca0f6078a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f629c30 0x55ca0f626160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa3ebfff640 0 client.4860 ms_handle_reset on v2:172.21.15.188:6832/3545719682 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f641710 0x55ca0f641b90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 --2- 172.21.15.160:0/1211141933 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f6082a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 >> 172.21.15.160:0/1211141933 conn(0x55ca0f6292f0 msgr2=0x55ca0f6277a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 shutdown_connections 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 1 -- 172.21.15.160:0/1211141933 wait complete. 2024-05-09T07:14:12.458 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.449+0000 7fa4163f0740 20 client.4860 trim_cache size 0 max 16384 2024-05-09T07:14:12.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 Processor -- start 2024-05-09T07:14:12.469 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 -- start start 2024-05-09T07:14:12.471 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f632ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f623cb0 0x55ca0f62ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6296d0 0x55ca0f6355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1f3ba0 con 0x55ca0f6292f0 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa40c1522a0 con 0x55ca0f6296d0 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa40c0ab400 con 0x55ca0f623cb0 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa411483640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f632ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa411483640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f632ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56002/0 (socket says 172.21.15.160:56002) 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.467+0000 7fa411483640 1 -- 172.21.15.160:0/2618089865 learned_addr learned my addr 172.21.15.160:0/2618089865 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa411483640 1 -- 172.21.15.160:0/2618089865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f623cb0 msgr2=0x55ca0f62ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa411483640 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f623cb0 0x55ca0f62ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa411483640 1 -- 172.21.15.160:0/2618089865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6296d0 msgr2=0x55ca0f6355c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa411483640 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6296d0 0x55ca0f6355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.472 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa411483640 1 -- 172.21.15.160:0/2618089865 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca0f62cb40 con 0x55ca0f6292f0 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa411483640 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f632ba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa408000c00 tx=0x7fa40802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f283b1657e2f9d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa3f67fc640 1 -- 172.21.15.160:0/2618089865 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408037040 con 0x55ca0f6292f0 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa3f67fc640 1 -- 172.21.15.160:0/2618089865 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408055070 con 0x55ca0f6292f0 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa3f67fc640 1 -- 172.21.15.160:0/2618089865 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408032ac0 con 0x55ca0f6292f0 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 -- 172.21.15.160:0/2618089865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 msgr2=0x55ca0f632ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f632ba0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa408000c00 tx=0x7fa40802c000 comp rx=0 tx=0).stop 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 -- 172.21.15.160:0/2618089865 shutdown_connections 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f6296d0 0x55ca0f6355c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f623cb0 0x55ca0f62ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 --2- 172.21.15.160:0/2618089865 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f632ba0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.468+0000 7fa4163f0740 1 -- 172.21.15.160:0/2618089865 >> 172.21.15.160:0/2618089865 conn(0x55ca0f623280 msgr2=0x55ca0f633bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.469+0000 7fa4163f0740 1 -- 172.21.15.160:0/2618089865 shutdown_connections 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.469+0000 7fa4163f0740 1 -- 172.21.15.160:0/2618089865 wait complete. 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.469+0000 7fa4163f0740 1 -- start start 2024-05-09T07:14:12.473 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.469+0000 7fa4163f0740 1 Processor -- start 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f623cb0 0x55ca0f6a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f608110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f60db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f623b30 con 0x55ca0f6292f0 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x55ca0f5eaba0 con 0x55ca0f623cb0 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x55ca0f624180 con 0x55ca0f6296d0 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa4163f0740 10 client.0 _ll_register_callbacks cb 0x7fa404017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T07:14:12.477 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f608110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f608110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56012/0 (socket says 172.21.15.160:56012) 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 -- 172.21.15.160:0/3532172980 learned_addr learned my addr 172.21.15.160:0/3532172980 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa411483640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f623cb0 0x55ca0f6a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa411c84640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f60db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 -- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 msgr2=0x55ca0f60db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f60db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 -- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f623cb0 msgr2=0x55ca0f6a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f623cb0 0x55ca0f6a8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 -- 172.21.15.160:0/3532172980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca0f6088a0 con 0x55ca0f6292f0 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa410c82640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f608110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc058270 tx=0x7fa3fc058dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82d0f822bf9c1de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.470+0000 7fa3ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2024-05-09T07:14:12.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa3ebfff640 1 -- 172.21.15.160:0/3532172980 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3fc070020 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa3ebfff640 1 -- 172.21.15.160:0/3532172980 <== mon.0 v2:172.21.15.160:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3fc0579c0 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa408055070 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa3ebfff640 1 -- 172.21.15.160:0/3532172980 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fa3fc057be0 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 10 client.4866 fetch_fsmap learned FSMap version 8 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ca0f5ee550 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa3ebfff640 1 -- 172.21.15.160:0/3532172980 <== mon.0 v2:172.21.15.160:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa408055070 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa3ebfff640 1 -- 172.21.15.160:0/3532172980 <== mon.0 v2:172.21.15.160:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fa3fc0542c0 con 0x55ca0f6292f0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 10 client.4866 fetch_fsmap finished waiting for FSMap version 8 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 10 client.4866 resolve_mds: resolved name 'd' to daemon mds.d 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 20 client.4866 populate_metadata read hostname 'smithi160' 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f629c30 0x55ca0f6a51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 4 client.4866 mds_command: new command op to 4296 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 --> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] -- command(tid 0: {"prefix": "status"}) -- 0x55ca0f4d5b80 con 0x55ca0f629c30 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa411483640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f629c30 0x55ca0f6a51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa3ebfff640 10 client.4866 ms_handle_connect on v2:172.21.15.188:6832/3545719682 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.471+0000 7fa411483640 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f629c30 0x55ca0f6a51a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c985f0cc4af35e2b server_cookie=2fba9111442a97d1 in_seq=0 out_seq=0 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa3ebfff640 1 -- 172.21.15.160:0/3532172980 <== mds.0 v2:172.21.15.188:6832/3545719682 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55ca0f4d5b80 con 0x55ca0f629c30 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa3ebfff640 10 client.4866 handle_command_reply: tid=0 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 client.4866 shutdown 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 msgr2=0x55ca0f608110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f608110 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc058270 tx=0x7fa3fc058dd0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 shutdown_connections 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:6832/3545719682,v1:172.21.15.188:6833/3545719682] conn(0x55ca0f629c30 0x55ca0f6a51a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x55ca0f6296d0 0x55ca0f60db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ca0f6292f0 0x55ca0f608110 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa3ebfff640 0 client.4866 ms_handle_reset on v2:172.21.15.188:6832/3545719682 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 --2- 172.21.15.160:0/3532172980 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x55ca0f623cb0 0x55ca0f6a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 >> 172.21.15.160:0/3532172980 conn(0x55ca0f623280 msgr2=0x55ca0f635bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.480 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 shutdown_connections 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 1 -- 172.21.15.160:0/3532172980 wait complete. 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.472+0000 7fa4163f0740 20 client.4866 trim_cache size 0 max 16384 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "cluster_fsid": "22e9edb5-902d-460d-aabf-1eff035c9624", 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "whoami": 0, 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 4296, 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "want_state": "up:active", 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "state": "up:active", 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs", 2024-05-09T07:14:12.481 INFO:teuthology.orchestra.run.smithi160.stdout: "rank_uptime": 27.000675335, 2024-05-09T07:14:12.482 INFO:teuthology.orchestra.run.smithi160.stdout: "mdsmap_epoch": 8, 2024-05-09T07:14:12.482 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch": 25, 2024-05-09T07:14:12.482 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_epoch_barrier": 25, 2024-05-09T07:14:12.482 INFO:teuthology.orchestra.run.smithi160.stdout: "uptime": 33.654955598999997 2024-05-09T07:14:12.482 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa3f006be30 msgr2=0x7fa3f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa3f006be30 0x7fa3f006e2f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa408034000 tx=0x7fa40802e000 comp rx=0 tx=0).stop 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 msgr2=0x7fa40c0c01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 0x7fa40c0c01a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00c430 tx=0x7fa3fc00c8f0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 shutdown_connections 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa3f006be30 0x7fa3f006e2f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa40c153300 0x7fa40c1ecd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa40c0c5e20 0x7fa40c0c01a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 --2- 172.21.15.160:0/4013252826 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa40c0a5fb0 0x7fa40c1e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 >> 172.21.15.160:0/4013252826 conn(0x7fa40c0c7580 msgr2=0x7fa40c0bc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 shutdown_connections 2024-05-09T07:14:12.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.473+0000 7fa4136e7640 1 -- 172.21.15.160:0/4013252826 wait complete. 2024-05-09T07:14:12.486 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T07:14:12.495 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T07:14:12.496 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T07:14:12.496 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T07:14:12.496 INFO:teuthology.run_tasks:Running task parallel... 2024-05-09T07:14:12.504 INFO:teuthology.task.parallel:starting parallel... 2024-05-09T07:14:12.504 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-09T07:14:12.504 INFO:teuthology.task.sequential:In sequential, running task ceph-fuse... 2024-05-09T07:14:12.506 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-05-09T07:14:12.506 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-05-09T07:14:12.506 INFO:tasks.ceph_fuse:client.0 config is {} 2024-05-09T07:14:12.506 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T07:14:12.506 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:12.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:12.506 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-05-09T07:14:12.507 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-05-09T07:14:12.507 INFO:teuthology.task.sequential:In sequential, running task kclient... 2024-05-09T07:14:12.510 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T07:14:12.510 INFO:tasks.kclient:config is {'client.1': {}} 2024-05-09T07:14:12.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:12.511 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-05-09T07:14:12.526 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-09T07:14:12.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:12.527 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-09T07:14:12.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:12.529 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-05-09T07:14:12.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:12.593 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2024-05-09T07:14:12.600 INFO:teuthology.orchestra.run.smithi188.stderr:Cannot find device "ceph-brx" 2024-05-09T07:14:12.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T07:14:12.602 INFO:tasks.kclient:client.1 config is {} 2024-05-09T07:14:12.602 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T07:14:12.602 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:12.602 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T07:14:12.702 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-05-09T07:14:12.702 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-05-09T07:14:12.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.735+0000 7f34311e7640 1 Processor -- start 2024-05-09T07:14:12.738 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.735+0000 7f34311e7640 1 -- start start 2024-05-09T07:14:12.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f34311e7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f342c153300 0x7f342c0c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f34311e7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f342c0a5fb0 0x7f342c0bbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.739 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f34311e7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342c1563b0 con 0x7f342c0c5e20 2024-05-09T07:14:12.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f34311e7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0c1630 con 0x7f342c0a5fb0 2024-05-09T07:14:12.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f34311e7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f342c0c19d0 con 0x7f342c153300 2024-05-09T07:14:12.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f342a59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f342c0a5fb0 0x7f342c0bbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f342b59e640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f342c0c5e20 0x7f342c0bc360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37494/0 (socket says 172.21.15.160:37494) 2024-05-09T07:14:12.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f342a59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f342c0a5fb0 0x7f342c0bbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56020/0 (socket says 172.21.15.160:56020) 2024-05-09T07:14:12.740 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f342b59e640 1 -- 172.21.15.160:0/3585690875 learned_addr learned my addr 172.21.15.160:0/3585690875 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f342a59c640 1 -- 172.21.15.160:0/3585690875 >> v2:172.21.15.188:3300/0 conn(0x7f342c153300 msgr2=0x7f342c0c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.736+0000 7f342ad9d640 1 --2- 172.21.15.160:0/3585690875 >> v2:172.21.15.188:3300/0 conn(0x7f342c153300 0x7f342c0c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f342a59c640 1 --2- 172.21.15.160:0/3585690875 >> v2:172.21.15.188:3300/0 conn(0x7f342c153300 0x7f342c0c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f3429d9b640 1 -- 172.21.15.160:0/3585690875 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 798622028 0 0) 0x7f342c1563b0 con 0x7f342c0c5e20 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f342a59c640 1 -- 172.21.15.160:0/3585690875 >> v1:172.21.15.160:6789/0 conn(0x7f342c0c5e20 legacy=0x7f342c0bc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f342a59c640 1 -- 172.21.15.160:0/3585690875 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c156690 con 0x7f342c0a5fb0 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f342ad9d640 1 --2- 172.21.15.160:0/3585690875 >> v2:172.21.15.188:3300/0 conn(0x7f342c153300 0x7f342c0c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f342a59c640 1 --2- 172.21.15.160:0/3585690875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0a5fb0 0x7f342c0bbe20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f342c06c1d0 tx=0x7f342002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40bc4d3311f0b2be server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f3429d9b640 1 -- 172.21.15.160:0/3585690875 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420037070 con 0x7f342c0a5fb0 2024-05-09T07:14:12.742 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f3429d9b640 1 -- 172.21.15.160:0/3585690875 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3420032440 con 0x7f342c0a5fb0 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.737+0000 7f3429d9b640 1 -- 172.21.15.160:0/3585690875 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420032810 con 0x7f342c0a5fb0 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 -- 172.21.15.160:0/3585690875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0a5fb0 msgr2=0x7f342c0bbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 --2- 172.21.15.160:0/3585690875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0a5fb0 0x7f342c0bbe20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f342c06c1d0 tx=0x7f342002a1a0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 -- 172.21.15.160:0/3585690875 shutdown_connections 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 --2- 172.21.15.160:0/3585690875 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0a5fb0 0x7f342c0bbe20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 --2- 172.21.15.160:0/3585690875 >> v2:172.21.15.188:3300/0 conn(0x7f342c153300 0x7f342c0c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 -- 172.21.15.160:0/3585690875 >> 172.21.15.160:0/3585690875 conn(0x7f342c0c7580 msgr2=0x7f342c0a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 -- 172.21.15.160:0/3585690875 shutdown_connections 2024-05-09T07:14:12.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.738+0000 7f34311e7640 1 -- 172.21.15.160:0/3585690875 wait complete. 2024-05-09T07:14:12.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 Processor -- start 2024-05-09T07:14:12.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 -- start start 2024-05-09T07:14:12.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f342c0a5fb0 0x7f342c151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 0x7f342c14fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f342c153300 0x7f342c150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0c2420 con 0x7f342c0c5e20 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f342c0c1f00 con 0x7f342c0a5fb0 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f34311e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f342c0bd220 con 0x7f342c153300 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f342a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 0x7f342c14fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f342a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 0x7f342c14fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56034/0 (socket says 172.21.15.160:56034) 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.740+0000 7f342a59c640 1 -- 172.21.15.160:0/1731903601 learned_addr learned my addr 172.21.15.160:0/1731903601 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342ad9d640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f342c0a5fb0 0x7f342c151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342b59e640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f342c153300 0x7f342c150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342a59c640 1 -- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f342c153300 msgr2=0x7f342c150230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342a59c640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f342c153300 0x7f342c150230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342a59c640 1 -- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f342c0a5fb0 msgr2=0x7f342c151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342a59c640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f342c0a5fb0 0x7f342c151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342a59c640 1 -- 172.21.15.160:0/1731903601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342c1eb9e0 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342b59e640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f342c153300 0x7f342c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342ad9d640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f342c0a5fb0 0x7f342c151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f342a59c640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 0x7f342c14fcf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3420000c00 tx=0x7f34200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420037040 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3420042070 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342c1ebc80 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3420030bd0 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.741+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342c1550c0 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.742+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f342c0756c0 con 0x7f342c0c5e20 2024-05-09T07:14:12.747 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.743+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3420047460 con 0x7f342c0c5e20 2024-05-09T07:14:12.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.743+0000 7f34137fe640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f341806be30 0x7f341806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.743+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f34200abed0 con 0x7f342c0c5e20 2024-05-09T07:14:12.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.743+0000 7f342ad9d640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f341806be30 0x7f341806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.748 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.744+0000 7f342ad9d640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f341806be30 0x7f341806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f341c00aa30 tx=0x7f341c008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.748+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3420032e20 con 0x7f342c0c5e20 2024-05-09T07:14:12.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.837+0000 7f3c1081c640 1 Processor -- start 2024-05-09T07:14:12.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.837+0000 7f3c1081c640 1 -- start start 2024-05-09T07:14:12.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c1081c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3c0c0c5610 0x7f3c0c0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c1081c640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c0c159840 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c1081c640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3c0c159560 con 0x7f3c0c0be320 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c1081c640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3c0c159310 con 0x7f3c0c0c5610 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c0a59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3c0c0c5610 0x7f3c0c0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c0a59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3c0c0c5610 0x7f3c0c0c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56042/0 (socket says 172.21.15.160:56042) 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c0a59c640 1 -- 172.21.15.160:0/1050360942 learned_addr learned my addr 172.21.15.160:0/1050360942 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.838+0000 7f3c0ad9d640 1 --1- 172.21.15.160:0/1050360942 >> v1:172.21.15.160:6789/0 conn(0x7f3c0c0c8b90 0x7f3c0c155a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:37504/0 (socket says 172.21.15.160:37504) 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.839+0000 7f3c0a59c640 1 -- 172.21.15.160:0/1050360942 >> v1:172.21.15.188:6789/0 conn(0x7f3c0c0be320 legacy=0x7f3c0c155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.839+0000 7f3c0a59c640 1 -- 172.21.15.160:0/1050360942 >> v1:172.21.15.160:6789/0 conn(0x7f3c0c0c8b90 legacy=0x7f3c0c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.839+0000 7f3c0a59c640 1 -- 172.21.15.160:0/1050360942 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c0c77d0 con 0x7f3c0c0c5610 2024-05-09T07:14:12.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.839+0000 7f3c0959a640 1 -- 172.21.15.160:0/1050360942 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 755569931 0 0) 0x7f3c0c159840 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.839+0000 7f3c0a59c640 1 --2- 172.21.15.160:0/1050360942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0c59f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009870 tx=0x7f3bfc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3d4181d63b38b8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c0959a640 1 -- 172.21.15.160:0/1050360942 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc037070 con 0x7f3c0c0c5610 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c0959a640 1 -- 172.21.15.160:0/1050360942 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3bfc032440 con 0x7f3c0c0c5610 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c0959a640 1 -- 172.21.15.160:0/1050360942 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bfc032810 con 0x7f3c0c0c5610 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 -- 172.21.15.160:0/1050360942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c5610 msgr2=0x7f3c0c0c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 --2- 172.21.15.160:0/1050360942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0c59f0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc009870 tx=0x7f3bfc02a180 comp rx=0 tx=0).stop 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 -- 172.21.15.160:0/1050360942 shutdown_connections 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 --2- 172.21.15.160:0/1050360942 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0c59f0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 -- 172.21.15.160:0/1050360942 >> 172.21.15.160:0/1050360942 conn(0x7f3c0c072e90 msgr2=0x7f3c0c0ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 -- 172.21.15.160:0/1050360942 shutdown_connections 2024-05-09T07:14:12.844 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.840+0000 7f3c1081c640 1 -- 172.21.15.160:0/1050360942 wait complete. 2024-05-09T07:14:12.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.841+0000 7f3c1081c640 1 Processor -- start 2024-05-09T07:14:12.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 -- start start 2024-05-09T07:14:12.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c0be320 0x7f3c0c1ebc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 0x7f3c0c1de970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0ad6a0 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c0c7d90 con 0x7f3c0c0c5610 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c1081c640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c152bb0 con 0x7f3c0c0be320 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c0ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 0x7f3c0c1de970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c0ad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 0x7f3c0c1de970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:56050/0 (socket says 172.21.15.160:56050) 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.842+0000 7f3c0ad9d640 1 -- 172.21.15.160:0/58650649 learned_addr learned my addr 172.21.15.160:0/58650649 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0a59c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c0be320 0x7f3c0c1ebc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c09d9b640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0ad9d640 1 -- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c0be320 msgr2=0x7f3c0c1ebc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0ad9d640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c0be320 0x7f3c0c1ebc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0ad9d640 1 -- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c0c5610 msgr2=0x7f3c0c0ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0ad9d640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0ad9d640 1 -- 172.21.15.160:0/58650649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c0c1400 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c0ad9d640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 0x7f3c0c1de970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c7a0 tx=0x7f3c0400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04018070 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c0400a640 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c0c16a0 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0400aa70 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.848 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.843+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c1eb1b0 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.844+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c0756c0 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.844+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3c0400ac10 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.845+0000 7f3c02ffd640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bec06be30 0x7f3bec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:14:12.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.845+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3c0408a810 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.845+0000 7f3c0a59c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bec06be30 0x7f3bec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:14:12.849 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.845+0000 7f3c0a59c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bec06be30 0x7f3bec06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:14:12.856 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.853+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3c04058e90 con 0x7f3c0c0c8b90 2024-05-09T07:14:12.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.972+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f342c0bc850 con 0x7f342c0c5e20 2024-05-09T07:14:12.975 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.972+0000 7f34137fe640 1 -- 172.21.15.160:0/1731903601 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f3420045360 con 0x7f342c0c5e20 2024-05-09T07:14:12.975 INFO:teuthology.orchestra.run.smithi160.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T07:14:12.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f341806be30 msgr2=0x7f341806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f341806be30 0x7f341806e2f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f341c00aa30 tx=0x7f341c008040 comp rx=0 tx=0).stop 2024-05-09T07:14:12.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 msgr2=0x7f342c14fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 0x7f342c14fcf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f3420000c00 tx=0x7f34200046c0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 shutdown_connections 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f341806be30 0x7f341806e2f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f342c153300 0x7f342c150230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f342c0c5e20 0x7f342c14fcf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 --2- 172.21.15.160:0/1731903601 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f342c0a5fb0 0x7f342c151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.974+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 >> 172.21.15.160:0/1731903601 conn(0x7f342c0c7580 msgr2=0x7f342c06d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.975+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 shutdown_connections 2024-05-09T07:14:12.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:12.975+0000 7f34311e7640 1 -- 172.21.15.160:0/1731903601 wait complete. 2024-05-09T07:14:12.988 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T07:14:12.988 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi188.front.sepia.ceph.com 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T07:14:12.989 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-05-09T07:14:12.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:12.989 DEBUG:teuthology.orchestra.run.smithi188:> ip addr 2024-05-09T07:14:13.008 INFO:teuthology.orchestra.run.smithi188.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T07:14:13.008 INFO:teuthology.orchestra.run.smithi188.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T07:14:13.008 INFO:teuthology.orchestra.run.smithi188.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 ::1/128 scope host 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:48 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f0 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:d9:37:49 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: altname enp7s0f1 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:38 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: link/ether 0c:c4:7a:88:80:39 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: inet 172.21.15.188/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T07:14:13.009 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-05-09T07:14:13.010 INFO:teuthology.orchestra.run.smithi188.stdout: inet6 fe80::ec4:7aff:fe88:8039/64 scope link 2024-05-09T07:14:13.010 INFO:teuthology.orchestra.run.smithi188.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:14:13.010 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add name ceph-brx type bridge 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr flush dev ceph-brx 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set ceph-brx up 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T07:14:13.010 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-05-09T07:14:13.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.072+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c0c0ad340 con 0x7f3c0c0c8b90 2024-05-09T07:14:13.075 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.072+0000 7f3c02ffd640 1 -- 172.21.15.160:0/58650649 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+88 (secure 0 0 0) 0x7f3c04088e20 con 0x7f3c0c0c8b90 2024-05-09T07:14:13.075 INFO:teuthology.orchestra.run.smithi160.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-09T07:14:13.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bec06be30 msgr2=0x7f3bec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:13.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bec06be30 0x7f3bec06e2f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3bfc034000 tx=0x7f3bfc02e000 comp rx=0 tx=0).stop 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 msgr2=0x7f3c0c1de970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 0x7f3c0c1de970 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3c0400c7a0 tx=0x7f3c0400cc60 comp rx=0 tx=0).stop 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 shutdown_connections 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3bec06be30 0x7f3bec06e2f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3c0c0c5610 0x7f3c0c0ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3c0c0c8b90 0x7f3c0c1de970 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 --2- 172.21.15.160:0/58650649 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3c0c0be320 0x7f3c0c1ebc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 >> 172.21.15.160:0/58650649 conn(0x7f3c0c072e90 msgr2=0x7f3c0c1e1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:14:13.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 shutdown_connections 2024-05-09T07:14:13.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13.074+0000 7f3c1081c640 1 -- 172.21.15.160:0/58650649 wait complete. 2024-05-09T07:14:13.088 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T07:14:13.088 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T07:14:13.088 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi160.front.sepia.ceph.com 2024-05-09T07:14:13.088 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T07:14:13.088 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:13.088 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T07:14:13.089 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T07:14:13.089 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T07:14:13.089 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T07:14:13.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.089 DEBUG:teuthology.orchestra.run.smithi160:> ip addr 2024-05-09T07:14:13.103 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.104 INFO:teuthology.orchestra.run.smithi160.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T07:14:13.104 INFO:teuthology.orchestra.run.smithi160.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T07:14:13.104 INFO:teuthology.orchestra.run.smithi160.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 ::1/128 scope host 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:80 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f0 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:d9:82:81 brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: altname enp7s0f1 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5a brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: link/ether 0c:c4:7a:88:7f:5b brd ff:ff:ff:ff:ff:ff 2024-05-09T07:14:13.105 INFO:teuthology.orchestra.run.smithi160.stdout: inet 172.21.15.160/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-05-09T07:14:13.106 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft 42748sec preferred_lft 42748sec 2024-05-09T07:14:13.106 INFO:teuthology.orchestra.run.smithi160.stdout: inet6 fe80::ec4:7aff:fe88:7f5b/64 scope link 2024-05-09T07:14:13.106 INFO:teuthology.orchestra.run.smithi160.stdout: valid_lft forever preferred_lft forever 2024-05-09T07:14:13.106 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T07:14:13.106 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.106 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:14:13.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add name ceph-brx type bridge 2024-05-09T07:14:13.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr flush dev ceph-brx 2024-05-09T07:14:13.106 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set ceph-brx up 2024-05-09T07:14:13.107 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T07:14:13.107 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:14:13.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.210 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.216 DEBUG:teuthology.orchestra.run.smithi188:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T07:14:13.239 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.241 INFO:teuthology.orchestra.run.smithi188.stdout:1 2024-05-09T07:14:13.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.247 DEBUG:teuthology.orchestra.run.smithi188:> route 2024-05-09T07:14:13.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.249 DEBUG:teuthology.orchestra.run.smithi160:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T07:14:13.274 INFO:teuthology.orchestra.run.smithi160.stdout:1 2024-05-09T07:14:13.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.276 DEBUG:teuthology.orchestra.run.smithi160:> route 2024-05-09T07:14:13.327 INFO:teuthology.orchestra.run.smithi188.stdout:Kernel IP routing table 2024-05-09T07:14:13.327 INFO:teuthology.orchestra.run.smithi188.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T07:14:13.328 INFO:teuthology.orchestra.run.smithi188.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T07:14:13.328 INFO:teuthology.orchestra.run.smithi188.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T07:14:13.328 INFO:teuthology.orchestra.run.smithi188.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T07:14:13.328 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.328 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-05-09T07:14:13.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T07:14:13.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T07:14:13.328 DEBUG:teuthology.orchestra.run.smithi188:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T07:14:13.328 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-05-09T07:14:13.333 INFO:teuthology.orchestra.run.smithi160.stdout:Kernel IP routing table 2024-05-09T07:14:13.333 INFO:teuthology.orchestra.run.smithi160.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T07:14:13.333 INFO:teuthology.orchestra.run.smithi160.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-05-09T07:14:13.333 INFO:teuthology.orchestra.run.smithi160.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-05-09T07:14:13.333 INFO:teuthology.orchestra.run.smithi160.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T07:14:13.334 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.334 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:14:13.334 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-05-09T07:14:13.334 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-05-09T07:14:13.334 DEBUG:teuthology.orchestra.run.smithi160:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-05-09T07:14:13.334 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:14:13.416 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.422 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.455 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.462 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-05-09T07:14:13.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.477 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list-id 2024-05-09T07:14:13.533 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.533 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:14:13.533 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T07:14:13.533 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T07:14:13.534 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:14:13.616 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.623 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-05-09T07:14:13.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:13.639 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list-id 2024-05-09T07:14:13.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.655 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T07:14:13.655 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.655 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:14:13.655 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T07:14:13.655 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T07:14:13.656 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T07:14:13.656 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T07:14:13.656 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T07:14:13.656 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:14:13.694 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.694 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-05-09T07:14:13.694 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T07:14:13.694 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-05-09T07:14:13.695 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-05-09T07:14:13.704 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.770 DEBUG:teuthology.orchestra.run.smithi160:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.770 DEBUG:teuthology.orchestra.run.smithi160:> set -e 2024-05-09T07:14:13.770 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set brx.0 up 2024-05-09T07:14:13.770 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T07:14:13.770 DEBUG:teuthology.orchestra.run.smithi160:> ') 2024-05-09T07:14:13.784 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.809 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.816 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-05-09T07:14:13.817 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-05-09T07:14:13.818 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-05-09T07:14:13.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.847 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-05-09T07:14:13.847 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T07:14:13.848 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:13.863 INFO:teuthology.orchestra.run.smithi160.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T07:14:13.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T07:14:13.864 DEBUG:teuthology.orchestra.run.smithi160:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:13.868 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:13.917 DEBUG:teuthology.orchestra.run.smithi160:> sudo modprobe fuse 2024-05-09T07:14:13.953 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:13.960 DEBUG:teuthology.orchestra.run.smithi188:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T07:14:13.960 DEBUG:teuthology.orchestra.run.smithi188:> set -e 2024-05-09T07:14:13.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set brx.0 up 2024-05-09T07:14:13.960 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T07:14:13.960 DEBUG:teuthology.orchestra.run.smithi188:> ') 2024-05-09T07:14:13.983 DEBUG:teuthology.orchestra.run.smithi160:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T07:14:14.009 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:14.031 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:14.038 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T07:14:14.038 DEBUG:teuthology.orchestra.run.smithi188:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/proc 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/sys 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/dev 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/security 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/shm 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/pts 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/run 2024-05-09T07:14:14.039 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/cgroup 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/pstore 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/bpf 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/ 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/selinux 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/hugepages 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/mqueue 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/debug 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/tracing 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/fuse/connections 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/config 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T07:14:14.040 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/1000 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T07:14:14.041 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T07:14:14.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.043 DEBUG:teuthology.orchestra.run.smithi160:> ls /sys/fs/fuse/connections 2024-05-09T07:14:14.053 INFO:teuthology.orchestra.run.smithi188.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-05-09T07:14:14.053 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T07:14:14.053 DEBUG:teuthology.orchestra.run.smithi188:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:14.098 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-09T07:14:14.098 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-05-09T07:14:14.104 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-05-09T07:14:14.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.104 DEBUG:teuthology.orchestra.run.smithi188:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T07:14:14.141 DEBUG:teuthology.orchestra.run.smithi160:> sudo modprobe fuse 2024-05-09T07:14:14.165 DEBUG:teuthology.orchestra.run.smithi160:> cat /proc/self/mounts | awk '{print $2}' 2024-05-09T07:14:14.196 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T07:14:14.196 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/proc 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/sys 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/dev 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/security 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/shm 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/pts 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/run 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/cgroup 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/pstore 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/bpf 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/ 2024-05-09T07:14:14.223 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/selinux 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/proc/sys/fs/binfmt_misc 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/hugepages 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/dev/mqueue 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/debug 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/tracing 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/fs/fuse/connections 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/sys/kernel/config 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-sysctl.service 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/nfs/rpc_pipefs 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/run/user/1000 2024-05-09T07:14:14.224 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-0 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-1 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-2 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/var/lib/ceph/osd/ceph-3 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run.smithi160.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T07:14:14.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.225 DEBUG:teuthology.orchestra.run.smithi160:> ls /sys/fs/fuse/connections 2024-05-09T07:14:14.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:2024-05-09T07:14:14.221+0000 7f1c6db17480 -1 init, newargv = 0x55d6e74da0c0 newargc=15 2024-05-09T07:14:14.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:ceph-fuse[46508]: starting ceph client 2024-05-09T07:14:14.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:ceph-fuse[46508]: starting fuse 2024-05-09T07:14:14.241 INFO:teuthology.orchestra.run.smithi160.stdout:40 2024-05-09T07:14:14.241 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-05-09T07:14:14.241 DEBUG:teuthology.orchestra.run.smithi160:> sudo stdin-killer -- python3 -c ' 2024-05-09T07:14:14.241 DEBUG:teuthology.orchestra.run.smithi160:> import glob 2024-05-09T07:14:14.241 DEBUG:teuthology.orchestra.run.smithi160:> import re 2024-05-09T07:14:14.241 DEBUG:teuthology.orchestra.run.smithi160:> import os 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> import subprocess 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> def _find_admin_socket(client_name): 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> files = glob.glob(asok_path) 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> # Given a non-glob path, it better be there 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> if "*" not in asok_path: 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> assert(len(files) == 1) 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> return files[0] 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> for f in files: 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-09T07:14:14.242 DEBUG:teuthology.orchestra.run.smithi160:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> contents = proc_f.read() 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> if mountpoint in contents: 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> return f 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> print(_find_admin_socket("client.0")) 2024-05-09T07:14:14.243 DEBUG:teuthology.orchestra.run.smithi160:> ' 2024-05-09T07:14:14.305 INFO:teuthology.orchestra.run.smithi188.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-05-09T07:14:14.305 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-05-09T07:14:14.305 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: resolved to: "172.21.15.160:3300,172.21.15.188:3300,172.21.15.188:3301" 2024-05-09T07:14:14.305 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: trying mount with new device syntax: 1@22e9edb5-902d-460d-aabf-1eff035c9624.cephfs=/ 2024-05-09T07:14:14.305 INFO:teuthology.orchestra.run.smithi188.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.160:3300/172.21.15.188:3300/172.21.15.188:3301" will pass to kernel 2024-05-09T07:14:14.306 INFO:teuthology.orchestra.run.smithi188.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-05-09T07:14:14.306 INFO:teuthology.orchestra.run.smithi188.stdout: You just mounted a file system that supports labels which does not 2024-05-09T07:14:14.306 INFO:teuthology.orchestra.run.smithi188.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T07:14:14.306 INFO:teuthology.orchestra.run.smithi188.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T07:14:14.306 INFO:teuthology.orchestra.run.smithi188.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T07:14:14.308 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T07:14:14.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.308 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:14.333 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T07:14:14.361 INFO:teuthology.orchestra.run.smithi160.stdout:/var/run/ceph/ceph-client.0.46508.asok 2024-05-09T07:14:14.364 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:14.373 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.46508.asok 2024-05-09T07:14:14.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.373 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.46508.asok status 2024-05-09T07:14:14.422 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T07:14:14.424 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:14.431 INFO:teuthology.orchestra.run.smithi188.stdout:client4568 2024-05-09T07:14:14.432 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:14:14.432 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:14:14.448 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T07:14:14.509 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:14:14.509 INFO:teuthology.orchestra.run.smithi160.stdout: "metadata": { 2024-05-09T07:14:14.509 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_sha1": "96805dcf909541c4ca9a1e90258de8ae5b6fa1bc", 2024-05-09T07:14:14.509 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 19.0.0-2515-g96805dcf (96805dcf909541c4ca9a1e90258de8ae5b6fa1bc) squid (dev)", 2024-05-09T07:14:14.509 INFO:teuthology.orchestra.run.smithi160.stdout: "entity_id": "0", 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "hostname": "smithi160", 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "pid": "46508", 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "root": "/" 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "dentry_count": 0, 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "dentry_pinned_count": 0, 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "id": 4893, 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "inst": { 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "name": { 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "client", 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "num": 4893 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:14:14.510 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": { 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "192.168.0.1:0", 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 3483451198 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: } 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": { 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "type": "v1", 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "addr": "192.168.0.1:0", 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "nonce": 3483451198 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "inst_str": "client.4893 192.168.0.1:0/3483451198", 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "addr_str": "192.168.0.1:0/3483451198", 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "inode_count": 1, 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_epoch": 8, 2024-05-09T07:14:14.511 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_epoch": 25, 2024-05-09T07:14:14.512 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_epoch_barrier": 0, 2024-05-09T07:14:14.512 INFO:teuthology.orchestra.run.smithi160.stdout: "blocklisted": false, 2024-05-09T07:14:14.512 INFO:teuthology.orchestra.run.smithi160.stdout: "fs_name": "cephfs" 2024-05-09T07:14:14.512 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:14:14.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.519 DEBUG:teuthology.orchestra.run.smithi160:> stat --file-system '--printf=%T 2024-05-09T07:14:14.519 DEBUG:teuthology.orchestra.run.smithi160:> ' -- /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:14.535 INFO:teuthology.orchestra.run.smithi160.stdout:fuseblk 2024-05-09T07:14:14.536 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:14.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:14:14.536 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:14.537 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T07:14:14.538 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:14.544 INFO:teuthology.orchestra.run.smithi188.stdout:client4568 2024-05-09T07:14:14.544 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T07:14:14.594 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T07:14:14.595 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:14.602 INFO:teuthology.orchestra.run.smithi188.stdout:client4568 2024-05-09T07:14:14.602 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/22e9edb5-902d-460d-aabf-1eff035c9624.client4568/status') 2024-05-09T07:14:14.603 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-09T07:14:14.606 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T07:14:14.607 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T07:14:14.607 DEBUG:teuthology.orchestra.run.smithi160:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:14.656 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 28h/40d Inode: 1 Links: 2 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:fusefs_t:s0 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2024-05-09 07:13:45.475487932 +0000 2024-05-09T07:14:14.659 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2024-05-09 07:13:54.937822174 +0000 2024-05-09T07:14:14.660 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2024-05-09T07:14:14.660 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T07:14:14.660 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T07:14:14.666 INFO:teuthology.orchestra.run.smithi188.stdout:instance: client.4568 (3)192.168.0.1:0/876735077 2024-05-09T07:14:14.666 INFO:teuthology.orchestra.run.smithi188.stdout:blocklisted: false 2024-05-09T07:14:14.666 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records in 2024-05-09T07:14:14.666 INFO:teuthology.orchestra.run.smithi188.stderr:0+1 records out 2024-05-09T07:14:14.666 INFO:teuthology.orchestra.run.smithi188.stderr:68 bytes copied, 2.3099e-05 s, 2.9 MB/s 2024-05-09T07:14:14.667 INFO:teuthology.orchestra.run.smithi188.stderr:2024-05-09T07:14:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T07:14:14.673 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-05-09T07:14:14.674 INFO:tasks.workunit:Pulling workunits from ref be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T07:14:14.674 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T07:14:14.674 DEBUG:teuthology.orchestra.run.smithi188:> stat -- /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:14.689 INFO:teuthology.orchestra.run.smithi188.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T07:14:14.689 INFO:teuthology.orchestra.run.smithi188.stdout:Device: 29h/41d Inode: 1 Links: 2 2024-05-09T07:14:14.689 INFO:teuthology.orchestra.run.smithi188.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T07:14:14.690 INFO:teuthology.orchestra.run.smithi188.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T07:14:14.690 INFO:teuthology.orchestra.run.smithi188.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-05-09T07:14:14.690 INFO:teuthology.orchestra.run.smithi188.stdout:Modify: 2024-05-09 07:13:45.475487932 +0000 2024-05-09T07:14:14.690 INFO:teuthology.orchestra.run.smithi188.stdout:Change: 2024-05-09 07:13:54.937822174 +0000 2024-05-09T07:14:14.690 INFO:teuthology.orchestra.run.smithi188.stdout: Birth: 2024-05-09 07:13:45.475487932 +0000 2024-05-09T07:14:14.690 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-05-09T07:14:14.691 DEBUG:teuthology.orchestra.run.smithi188:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-05-09T07:14:14.728 INFO:tasks.workunit:timeout=3h 2024-05-09T07:14:14.728 INFO:tasks.workunit:cleanup=True 2024-05-09T07:14:14.729 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T07:14:14.758 INFO:tasks.workunit:timeout=3h 2024-05-09T07:14:14.758 INFO:tasks.workunit:cleanup=True 2024-05-09T07:14:14.759 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout be85c202969d25af6e130678e7d73a20e9c4ca11 2024-05-09T07:14:14.785 INFO:tasks.workunit.client.0.smithi160.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T07:14:14.815 INFO:tasks.workunit.client.1.smithi188.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-05-09T07:14:15.603 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:14:15.606 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:14:45.678 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:14:45.681 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:15:15.743 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:15:15.746 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:15:25.443 INFO:tasks.workunit.client.0.smithi160.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T07:15:25.443 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: git switch -c 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:Or undo this operation with: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: git switch - 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.444 INFO:tasks.workunit.client.0.smithi160.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T07:15:25.445 INFO:tasks.workunit.client.0.smithi160.stderr: 2024-05-09T07:15:25.445 INFO:tasks.workunit.client.0.smithi160.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T07:15:25.445 DEBUG:teuthology.orchestra.run.smithi160:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T07:15:25.529 INFO:tasks.workunit.client.0.smithi160.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T07:15:25.531 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T07:15:25.531 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T07:15:26.093 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T07:15:26.149 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T07:15:26.190 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T07:15:26.192 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T07:15:26.192 INFO:tasks.workunit.client.0.smithi160.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T07:15:26.231 INFO:tasks.workunit.client.0.smithi160.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T07:15:26.234 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:15:26.234 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T07:15:26.250 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-05-09T07:15:26.251 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-05-09T07:15:26.252 DEBUG:teuthology.orchestra.run.smithi160:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-05-09T07:15:26.316 INFO:tasks.workunit.client.0.smithi160.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Iozone: Performance Test of File I/O 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Version $Revision: 3.494 $ 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Compiled for 64 bit mode. 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Build: linux-AMD64 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T07:15:26.388 INFO:tasks.workunit.client.0.smithi160.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Run began: Thu May 9 07:15:26 2024 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Include close in write timing 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Include fsync in write timing 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: File size set to 1048576 kB 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Record Size 16 kB 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Output is in kBytes/sec 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T07:15:26.389 INFO:tasks.workunit.client.0.smithi160.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T07:15:26.390 INFO:tasks.workunit.client.0.smithi160.stdout: Processor cache line size set to 32 bytes. 2024-05-09T07:15:26.390 INFO:tasks.workunit.client.0.smithi160.stdout: File stride size set to 17 * record size. 2024-05-09T07:15:26.390 INFO:tasks.workunit.client.0.smithi160.stdout: Throughput test with 1 process 2024-05-09T07:15:26.390 INFO:tasks.workunit.client.0.smithi160.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr:Note: switching to 'be85c202969d25af6e130678e7d73a20e9c4ca11'. 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T07:15:26.915 INFO:tasks.workunit.client.1.smithi188.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: git switch -c 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr:Or undo this operation with: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: git switch - 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr: 2024-05-09T07:15:26.916 INFO:tasks.workunit.client.1.smithi188.stderr:HEAD is now at be85c202969 Merge pull request #57242 from cbodley/wip-65767-squid 2024-05-09T07:15:26.919 DEBUG:teuthology.orchestra.run.smithi188:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-05-09T07:15:26.999 INFO:tasks.workunit.client.1.smithi188.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T07:15:27.001 INFO:tasks.workunit.client.1.smithi188.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-09T07:15:27.001 INFO:tasks.workunit.client.1.smithi188.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T07:15:27.606 INFO:tasks.workunit.client.1.smithi188.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T07:15:27.667 INFO:tasks.workunit.client.1.smithi188.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T07:15:27.713 INFO:tasks.workunit.client.1.smithi188.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-05-09T07:15:27.714 INFO:tasks.workunit.client.1.smithi188.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-09T07:15:27.714 INFO:tasks.workunit.client.1.smithi188.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T07:15:27.755 INFO:tasks.workunit.client.1.smithi188.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-05-09T07:15:27.758 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:15:27.759 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-05-09T07:15:27.774 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.1... 2024-05-09T07:15:27.775 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-05-09T07:15:27.775 DEBUG:teuthology.orchestra.run.smithi188:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=be85c202969d25af6e130678e7d73a20e9c4ca11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/dbench.sh 2024-05-09T07:15:27.837 INFO:tasks.workunit.client.1.smithi188.stderr:+ dbench 1 2024-05-09T07:15:27.850 INFO:tasks.workunit.client.1.smithi188.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-09T07:15:27.851 INFO:tasks.workunit.client.1.smithi188.stdout: 2024-05-09T07:15:27.851 INFO:tasks.workunit.client.1.smithi188.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-09T07:15:27.853 INFO:tasks.workunit.client.1.smithi188.stdout:0 of 1 processes prepared for launch 0 sec 2024-05-09T07:15:27.954 INFO:tasks.workunit.client.1.smithi188.stdout:1 of 1 processes prepared for launch 0 sec 2024-05-09T07:15:27.954 INFO:tasks.workunit.client.1.smithi188.stdout:releasing clients 2024-05-09T07:15:28.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 3548 38.42 MB/sec warmup 1 sec latency 14.074 ms 2024-05-09T07:15:29.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 6548 26.32 MB/sec warmup 2 sec latency 13.748 ms 2024-05-09T07:15:30.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 9693 22.41 MB/sec warmup 3 sec latency 12.124 ms 2024-05-09T07:15:31.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 12353 20.90 MB/sec warmup 4 sec latency 24.921 ms 2024-05-09T07:15:32.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 15288 19.88 MB/sec warmup 5 sec latency 17.082 ms 2024-05-09T07:15:33.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 18656 19.55 MB/sec warmup 6 sec latency 11.985 ms 2024-05-09T07:15:34.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 21991 19.41 MB/sec warmup 7 sec latency 27.452 ms 2024-05-09T07:15:35.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 25405 19.36 MB/sec warmup 8 sec latency 11.024 ms 2024-05-09T07:15:36.352 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:15:36.352 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 initial writers = 105260.64 kB/sec 2024-05-09T07:15:36.352 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 initial writers = 105258.79 kB/sec 2024-05-09T07:15:36.352 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 105260.64 kB/sec 2024-05-09T07:15:36.352 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 105260.64 kB/sec 2024-05-09T07:15:36.353 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 105260.64 kB/sec 2024-05-09T07:15:36.353 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:15:36.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 29078 19.34 MB/sec warmup 9 sec latency 12.275 ms 2024-05-09T07:15:37.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 32697 19.34 MB/sec warmup 10 sec latency 34.372 ms 2024-05-09T07:15:38.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 36299 19.33 MB/sec warmup 11 sec latency 14.152 ms 2024-05-09T07:15:39.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 40163 19.44 MB/sec warmup 12 sec latency 12.480 ms 2024-05-09T07:15:40.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 43587 19.34 MB/sec warmup 13 sec latency 40.457 ms 2024-05-09T07:15:41.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 47635 19.62 MB/sec warmup 14 sec latency 11.634 ms 2024-05-09T07:15:42.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 51349 19.60 MB/sec warmup 15 sec latency 28.350 ms 2024-05-09T07:15:43.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 54416 19.37 MB/sec warmup 16 sec latency 17.306 ms 2024-05-09T07:15:44.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 57737 19.28 MB/sec warmup 17 sec latency 14.083 ms 2024-05-09T07:15:45.832 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:15:45.834 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:15:45.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 61070 19.24 MB/sec warmup 18 sec latency 18.465 ms 2024-05-09T07:15:46.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 64477 19.22 MB/sec warmup 19 sec latency 14.408 ms 2024-05-09T07:15:47.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 67805 19.21 MB/sec warmup 20 sec latency 13.365 ms 2024-05-09T07:15:48.957 INFO:tasks.workunit.client.1.smithi188.stdout: 1 71178 19.20 MB/sec warmup 21 sec latency 28.059 ms 2024-05-09T07:15:49.954 INFO:tasks.workunit.client.1.smithi188.stdout: 1 74508 19.13 MB/sec warmup 22 sec latency 11.777 ms 2024-05-09T07:15:50.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 77524 18.92 MB/sec warmup 23 sec latency 12.152 ms 2024-05-09T07:15:51.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 80227 18.64 MB/sec warmup 24 sec latency 26.356 ms 2024-05-09T07:15:52.065 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:15:52.065 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 rewriters = 112913.58 kB/sec 2024-05-09T07:15:52.065 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 rewriters = 112911.71 kB/sec 2024-05-09T07:15:52.065 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 112913.58 kB/sec 2024-05-09T07:15:52.065 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 112913.58 kB/sec 2024-05-09T07:15:52.066 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 112913.58 kB/sec 2024-05-09T07:15:52.066 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:15:52.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 83971 18.71 MB/sec warmup 25 sec latency 11.920 ms 2024-05-09T07:15:53.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 86880 18.66 MB/sec warmup 26 sec latency 15.632 ms 2024-05-09T07:15:54.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 90704 18.69 MB/sec warmup 27 sec latency 14.045 ms 2024-05-09T07:15:55.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 94712 18.76 MB/sec warmup 28 sec latency 18.540 ms 2024-05-09T07:15:56.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 98737 18.96 MB/sec warmup 29 sec latency 11.833 ms 2024-05-09T07:15:57.820 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:15:57.820 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 readers = 7280969.50 kB/sec 2024-05-09T07:15:57.820 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 readers = 7267650.75 kB/sec 2024-05-09T07:15:57.820 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 7280969.50 kB/sec 2024-05-09T07:15:57.820 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 7280969.50 kB/sec 2024-05-09T07:15:57.820 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 7280969.50 kB/sec 2024-05-09T07:15:57.821 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:15:57.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 101962 18.82 MB/sec warmup 30 sec latency 17.006 ms 2024-05-09T07:15:58.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 104064 18.63 MB/sec warmup 31 sec latency 172.755 ms 2024-05-09T07:15:59.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 104379 18.15 MB/sec warmup 32 sec latency 418.145 ms 2024-05-09T07:16:00.290 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:00.291 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 re-readers = 6542358.00 kB/sec 2024-05-09T07:16:00.291 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 re-readers = 6534892.17 kB/sec 2024-05-09T07:16:00.291 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 6542358.00 kB/sec 2024-05-09T07:16:00.291 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 6542358.00 kB/sec 2024-05-09T07:16:00.291 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 6542358.00 kB/sec 2024-05-09T07:16:00.291 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:16:00.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 106862 18.02 MB/sec warmup 33 sec latency 27.840 ms 2024-05-09T07:16:01.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 110252 18.04 MB/sec warmup 34 sec latency 26.681 ms 2024-05-09T07:16:02.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 113506 18.04 MB/sec warmup 35 sec latency 37.145 ms 2024-05-09T07:16:03.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 116879 17.98 MB/sec warmup 36 sec latency 21.510 ms 2024-05-09T07:16:04.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 119985 17.94 MB/sec warmup 37 sec latency 40.940 ms 2024-05-09T07:16:05.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 123526 17.97 MB/sec warmup 38 sec latency 21.242 ms 2024-05-09T07:16:06.955 INFO:tasks.workunit.client.1.smithi188.stdout: 1 126637 17.88 MB/sec warmup 39 sec latency 61.666 ms 2024-05-09T07:16:07.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 129365 17.87 MB/sec warmup 40 sec latency 19.626 ms 2024-05-09T07:16:08.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 132925 17.90 MB/sec warmup 41 sec latency 13.345 ms 2024-05-09T07:16:09.220 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:09.220 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:09.220 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:09.220 INFO:tasks.workunit.client.0.smithi160.stdout:iozone test complete. 2024-05-09T07:16:09.220 INFO:tasks.workunit.client.0.smithi160.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: Iozone: Performance Test of File I/O 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: Version $Revision: 3.494 $ 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: Compiled for 64 bit mode. 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: Build: linux-AMD64 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T07:16:09.288 INFO:tasks.workunit.client.0.smithi160.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Run began: Thu May 9 07:16:09 2024 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Include close in write timing 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Include fsync in write timing 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: File size set to 1048576 kB 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Record Size 1024 kB 2024-05-09T07:16:09.289 INFO:tasks.workunit.client.0.smithi160.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: Output is in kBytes/sec 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: Processor cache line size set to 32 bytes. 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: File stride size set to 17 * record size. 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: Throughput test with 1 process 2024-05-09T07:16:09.290 INFO:tasks.workunit.client.0.smithi160.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-05-09T07:16:09.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 135377 17.80 MB/sec warmup 42 sec latency 39.144 ms 2024-05-09T07:16:10.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 136856 17.52 MB/sec warmup 43 sec latency 60.825 ms 2024-05-09T07:16:11.234 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:11.235 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 initial writers = 547661.62 kB/sec 2024-05-09T07:16:11.235 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 initial writers = 547577.02 kB/sec 2024-05-09T07:16:11.235 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 547661.62 kB/sec 2024-05-09T07:16:11.235 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 547661.62 kB/sec 2024-05-09T07:16:11.235 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 547661.62 kB/sec 2024-05-09T07:16:11.235 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:16:11.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 139440 17.47 MB/sec warmup 44 sec latency 102.255 ms 2024-05-09T07:16:12.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 142980 17.51 MB/sec warmup 45 sec latency 33.271 ms 2024-05-09T07:16:13.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 146518 17.54 MB/sec warmup 46 sec latency 15.363 ms 2024-05-09T07:16:14.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 148753 17.43 MB/sec warmup 47 sec latency 31.651 ms 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 rewriters = 582030.62 kB/sec 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 rewriters = 581953.50 kB/sec 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 582030.62 kB/sec 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 582030.62 kB/sec 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 582030.62 kB/sec 2024-05-09T07:16:15.878 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:16:15.906 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:16:15.908 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:16:15.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 149769 17.18 MB/sec warmup 48 sec latency 28.392 ms 2024-05-09T07:16:16.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 153310 17.22 MB/sec warmup 49 sec latency 21.528 ms 2024-05-09T07:16:17.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 156825 17.25 MB/sec warmup 50 sec latency 13.973 ms 2024-05-09T07:16:18.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 160306 17.29 MB/sec warmup 51 sec latency 20.821 ms 2024-05-09T07:16:19.678 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:19.678 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 readers = 7643568.00 kB/sec 2024-05-09T07:16:19.678 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 readers = 7632333.10 kB/sec 2024-05-09T07:16:19.678 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 7643568.00 kB/sec 2024-05-09T07:16:19.678 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 7643568.00 kB/sec 2024-05-09T07:16:19.679 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 7643568.00 kB/sec 2024-05-09T07:16:19.679 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:16:19.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 161041 17.05 MB/sec warmup 52 sec latency 257.016 ms 2024-05-09T07:16:20.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 163923 17.00 MB/sec warmup 53 sec latency 26.491 ms 2024-05-09T07:16:21.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 167467 17.04 MB/sec warmup 54 sec latency 24.680 ms 2024-05-09T07:16:22.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 170939 17.07 MB/sec warmup 55 sec latency 18.831 ms 2024-05-09T07:16:23.012 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:23.013 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 re-readers = 7496310.50 kB/sec 2024-05-09T07:16:23.013 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 re-readers = 7482345.68 kB/sec 2024-05-09T07:16:23.013 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 7496310.50 kB/sec 2024-05-09T07:16:23.013 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 7496310.50 kB/sec 2024-05-09T07:16:23.013 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 7496310.50 kB/sec 2024-05-09T07:16:23.013 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 1048576.00 kB 2024-05-09T07:16:23.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 174521 17.11 MB/sec warmup 56 sec latency 16.103 ms 2024-05-09T07:16:24.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 178082 17.15 MB/sec warmup 57 sec latency 28.223 ms 2024-05-09T07:16:25.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 181629 17.18 MB/sec warmup 58 sec latency 21.186 ms 2024-05-09T07:16:26.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 185172 17.21 MB/sec warmup 59 sec latency 22.585 ms 2024-05-09T07:16:27.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 188676 17.24 MB/sec warmup 60 sec latency 38.843 ms 2024-05-09T07:16:28.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 192223 17.28 MB/sec warmup 61 sec latency 20.442 ms 2024-05-09T07:16:29.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 195503 17.30 MB/sec warmup 62 sec latency 19.428 ms 2024-05-09T07:16:30.504 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:30.504 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:30.504 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:30.504 INFO:tasks.workunit.client.0.smithi160.stdout:iozone test complete. 2024-05-09T07:16:30.505 INFO:tasks.workunit.client.0.smithi160.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-09T07:16:30.572 INFO:tasks.workunit.client.0.smithi160.stdout: Iozone: Performance Test of File I/O 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Version $Revision: 3.494 $ 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Compiled for 64 bit mode. 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Build: linux-AMD64 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T07:16:30.573 INFO:tasks.workunit.client.0.smithi160.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Run began: Thu May 9 07:16:30 2024 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Include close in write timing 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Include fsync in write timing 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: File size set to 10485760 kB 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Record Size 1024 kB 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Output is in kBytes/sec 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Processor cache line size set to 32 bytes. 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: File stride size set to 17 * record size. 2024-05-09T07:16:30.574 INFO:tasks.workunit.client.0.smithi160.stdout: Throughput test with 1 process 2024-05-09T07:16:30.575 INFO:tasks.workunit.client.0.smithi160.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-05-09T07:16:30.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 198420 17.26 MB/sec warmup 63 sec latency 28.623 ms 2024-05-09T07:16:31.956 INFO:tasks.workunit.client.1.smithi188.stdout: 1 199428 17.07 MB/sec warmup 64 sec latency 24.444 ms 2024-05-09T07:16:32.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 201107 16.92 MB/sec warmup 65 sec latency 23.424 ms 2024-05-09T07:16:33.957 INFO:tasks.workunit.client.1.smithi188.stdout: 1 202575 16.83 MB/sec warmup 66 sec latency 20.350 ms 2024-05-09T07:16:34.957 INFO:tasks.workunit.client.1.smithi188.stdout: 1 203596 16.66 MB/sec warmup 67 sec latency 34.815 ms 2024-05-09T07:16:35.957 INFO:tasks.workunit.client.1.smithi188.stdout: 1 205523 16.55 MB/sec warmup 68 sec latency 31.525 ms 2024-05-09T07:16:36.957 INFO:tasks.workunit.client.1.smithi188.stdout: 1 206406 16.38 MB/sec warmup 69 sec latency 63.659 ms 2024-05-09T07:16:37.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 207989 16.23 MB/sec warmup 70 sec latency 28.194 ms 2024-05-09T07:16:38.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 209491 16.18 MB/sec warmup 71 sec latency 34.685 ms 2024-05-09T07:16:39.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 210235 15.97 MB/sec warmup 72 sec latency 27.289 ms 2024-05-09T07:16:40.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 210704 15.81 MB/sec warmup 73 sec latency 204.000 ms 2024-05-09T07:16:41.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 212264 15.70 MB/sec warmup 74 sec latency 36.041 ms 2024-05-09T07:16:42.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 213495 15.58 MB/sec warmup 75 sec latency 27.550 ms 2024-05-09T07:16:43.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 215221 15.46 MB/sec warmup 76 sec latency 32.235 ms 2024-05-09T07:16:44.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 216723 15.42 MB/sec warmup 77 sec latency 29.474 ms 2024-05-09T07:16:45.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 217910 15.29 MB/sec warmup 78 sec latency 23.957 ms 2024-05-09T07:16:46.188 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:16:46.190 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:16:46.958 INFO:tasks.workunit.client.1.smithi188.stdout: 1 219948 15.25 MB/sec warmup 79 sec latency 36.918 ms 2024-05-09T07:16:47.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 220755 15.09 MB/sec warmup 80 sec latency 51.963 ms 2024-05-09T07:16:48.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 222742 15.03 MB/sec warmup 81 sec latency 23.279 ms 2024-05-09T07:16:49.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 224235 14.95 MB/sec warmup 82 sec latency 35.104 ms 2024-05-09T07:16:50.278 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:16:50.279 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 initial writers = 533019.56 kB/sec 2024-05-09T07:16:50.279 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 initial writers = 533010.48 kB/sec 2024-05-09T07:16:50.279 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 533019.56 kB/sec 2024-05-09T07:16:50.279 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 533019.56 kB/sec 2024-05-09T07:16:50.279 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 533019.56 kB/sec 2024-05-09T07:16:50.279 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 10485760.00 kB 2024-05-09T07:16:50.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 227359 14.99 MB/sec warmup 83 sec latency 36.065 ms 2024-05-09T07:16:51.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 230839 15.04 MB/sec warmup 84 sec latency 39.153 ms 2024-05-09T07:16:52.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 234220 15.08 MB/sec warmup 85 sec latency 30.461 ms 2024-05-09T07:16:53.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 237450 15.09 MB/sec warmup 86 sec latency 20.298 ms 2024-05-09T07:16:54.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 240716 15.11 MB/sec warmup 87 sec latency 15.694 ms 2024-05-09T07:16:55.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 244158 15.16 MB/sec warmup 88 sec latency 20.134 ms 2024-05-09T07:16:56.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 246426 15.12 MB/sec warmup 89 sec latency 104.146 ms 2024-05-09T07:16:57.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 249938 15.16 MB/sec warmup 90 sec latency 20.137 ms 2024-05-09T07:16:58.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 253584 15.21 MB/sec warmup 91 sec latency 14.577 ms 2024-05-09T07:16:59.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 255761 15.19 MB/sec warmup 92 sec latency 28.365 ms 2024-05-09T07:17:00.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 256228 15.04 MB/sec warmup 93 sec latency 334.253 ms 2024-05-09T07:17:01.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 257377 14.93 MB/sec warmup 94 sec latency 155.439 ms 2024-05-09T07:17:02.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 258793 14.87 MB/sec warmup 95 sec latency 66.412 ms 2024-05-09T07:17:03.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 259711 14.76 MB/sec warmup 96 sec latency 45.419 ms 2024-05-09T07:17:04.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 261386 14.68 MB/sec warmup 97 sec latency 34.212 ms 2024-05-09T07:17:05.959 INFO:tasks.workunit.client.1.smithi188.stdout: 1 262930 14.65 MB/sec warmup 98 sec latency 28.324 ms 2024-05-09T07:17:06.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 264031 14.56 MB/sec warmup 99 sec latency 50.168 ms 2024-05-09T07:17:07.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 266273 14.55 MB/sec warmup 100 sec latency 24.193 ms 2024-05-09T07:17:08.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 267168 14.43 MB/sec warmup 101 sec latency 27.328 ms 2024-05-09T07:17:09.962 INFO:tasks.workunit.client.1.smithi188.stdout: 1 269409 14.41 MB/sec warmup 102 sec latency 32.415 ms 2024-05-09T07:17:10.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 270403 14.32 MB/sec warmup 103 sec latency 39.634 ms 2024-05-09T07:17:11.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 272038 14.24 MB/sec warmup 104 sec latency 94.574 ms 2024-05-09T07:17:12.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 273573 14.22 MB/sec warmup 105 sec latency 26.888 ms 2024-05-09T07:17:13.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 274681 14.14 MB/sec warmup 106 sec latency 33.327 ms 2024-05-09T07:17:14.960 INFO:tasks.workunit.client.1.smithi188.stdout: 1 276487 14.09 MB/sec warmup 107 sec latency 26.068 ms 2024-05-09T07:17:15.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 277379 14.01 MB/sec warmup 108 sec latency 29.147 ms 2024-05-09T07:17:16.253 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:17:16.259 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:17:16.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 278103 13.92 MB/sec warmup 109 sec latency 281.825 ms 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 rewriters = 553799.19 kB/sec 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 rewriters = 553793.10 kB/sec 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 553799.19 kB/sec 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 553799.19 kB/sec 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 553799.19 kB/sec 2024-05-09T07:17:17.896 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 10485760.00 kB 2024-05-09T07:17:17.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 279289 13.82 MB/sec warmup 110 sec latency 77.803 ms 2024-05-09T07:17:18.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 282692 13.86 MB/sec warmup 111 sec latency 21.103 ms 2024-05-09T07:17:19.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 286170 13.91 MB/sec warmup 112 sec latency 20.475 ms 2024-05-09T07:17:20.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 289728 13.95 MB/sec warmup 113 sec latency 19.162 ms 2024-05-09T07:17:21.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 293304 14.00 MB/sec warmup 114 sec latency 29.401 ms 2024-05-09T07:17:22.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 296887 14.05 MB/sec warmup 115 sec latency 18.961 ms 2024-05-09T07:17:23.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 300321 14.09 MB/sec warmup 116 sec latency 23.867 ms 2024-05-09T07:17:24.961 INFO:tasks.workunit.client.1.smithi188.stdout: 1 303585 14.12 MB/sec warmup 117 sec latency 11.976 ms 2024-05-09T07:17:25.962 INFO:tasks.workunit.client.1.smithi188.stdout: 1 306926 14.16 MB/sec warmup 118 sec latency 16.479 ms 2024-05-09T07:17:26.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 309872 14.17 MB/sec warmup 119 sec latency 54.281 ms 2024-05-09T07:17:28.962 INFO:tasks.workunit.client.1.smithi188.stdout: 1 316777 19.02 MB/sec execute 1 sec latency 23.585 ms 2024-05-09T07:17:29.962 INFO:tasks.workunit.client.1.smithi188.stdout: 1 320382 19.12 MB/sec execute 2 sec latency 19.835 ms 2024-05-09T07:17:30.962 INFO:tasks.workunit.client.1.smithi188.stdout: 1 323867 19.09 MB/sec execute 3 sec latency 18.049 ms 2024-05-09T07:17:31.962 INFO:tasks.workunit.client.1.smithi188.stdout: 1 327251 18.79 MB/sec execute 4 sec latency 16.263 ms 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 readers = 7623961.00 kB/sec 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 readers = 7622460.09 kB/sec 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 7623961.00 kB/sec 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 7623961.00 kB/sec 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 7623961.00 kB/sec 2024-05-09T07:17:32.855 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 10485760.00 kB 2024-05-09T07:17:32.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 330608 18.76 MB/sec execute 5 sec latency 21.930 ms 2024-05-09T07:17:33.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 334048 18.78 MB/sec execute 6 sec latency 16.341 ms 2024-05-09T07:17:34.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 337534 18.83 MB/sec execute 7 sec latency 23.233 ms 2024-05-09T07:17:35.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 340814 18.82 MB/sec execute 8 sec latency 30.441 ms 2024-05-09T07:17:36.735 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:17:36.736 INFO:tasks.workunit.client.0.smithi160.stdout: Children see throughput for 1 re-readers = 7500167.50 kB/sec 2024-05-09T07:17:36.736 INFO:tasks.workunit.client.0.smithi160.stdout: Parent sees throughput for 1 re-readers = 7498655.02 kB/sec 2024-05-09T07:17:36.736 INFO:tasks.workunit.client.0.smithi160.stdout: Min throughput per process = 7500167.50 kB/sec 2024-05-09T07:17:36.736 INFO:tasks.workunit.client.0.smithi160.stdout: Max throughput per process = 7500167.50 kB/sec 2024-05-09T07:17:36.736 INFO:tasks.workunit.client.0.smithi160.stdout: Avg throughput per process = 7500167.50 kB/sec 2024-05-09T07:17:36.736 INFO:tasks.workunit.client.0.smithi160.stdout: Min xfer = 10485760.00 kB 2024-05-09T07:17:36.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 344047 18.70 MB/sec execute 9 sec latency 22.455 ms 2024-05-09T07:17:37.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 347554 18.74 MB/sec execute 10 sec latency 24.971 ms 2024-05-09T07:17:38.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 350968 18.51 MB/sec execute 11 sec latency 15.132 ms 2024-05-09T07:17:39.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 354175 18.40 MB/sec execute 12 sec latency 16.077 ms 2024-05-09T07:17:40.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 357201 18.09 MB/sec execute 13 sec latency 15.613 ms 2024-05-09T07:17:41.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 358916 17.62 MB/sec execute 14 sec latency 159.175 ms 2024-05-09T07:17:42.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 361168 17.27 MB/sec execute 15 sec latency 264.073 ms 2024-05-09T07:17:43.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 362458 16.61 MB/sec execute 16 sec latency 37.502 ms 2024-05-09T07:17:44.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 363685 15.94 MB/sec execute 17 sec latency 22.298 ms 2024-05-09T07:17:45.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 365409 15.74 MB/sec execute 18 sec latency 28.415 ms 2024-05-09T07:17:46.596 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:17:46.598 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:17:46.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 367741 15.36 MB/sec execute 19 sec latency 19.482 ms 2024-05-09T07:17:47.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 371068 15.48 MB/sec execute 20 sec latency 29.203 ms 2024-05-09T07:17:48.289 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:17:48.289 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:17:48.289 INFO:tasks.workunit.client.0.smithi160.stdout: 2024-05-09T07:17:48.289 INFO:tasks.workunit.client.0.smithi160.stdout:iozone test complete. 2024-05-09T07:17:48.290 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T07:17:48.290 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T07:17:48.323 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-05-09T07:17:48.323 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T07:17:48.657 DEBUG:teuthology.parallel:result is None 2024-05-09T07:17:48.658 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T07:17:48.696 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T07:17:48.696 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-05-09T07:17:48.696 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:17:48.696 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:17:48.752 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:17:48.752 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:17:48.810 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:17:48.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.950+0000 7f6769d90640 1 Processor -- start 2024-05-09T07:17:48.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.951+0000 7f6769d90640 1 -- start start 2024-05-09T07:17:48.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.951+0000 7f6769d90640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67640be320 0x7f67640c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.951+0000 7f6769d90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f6764155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.951+0000 7f6769d90640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6764159840 con 0x7f67640c5610 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.951+0000 7f6769d90640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f6764159310 con 0x7f67640be320 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.951+0000 7f6769d90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6764159560 con 0x7f67640c8b90 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67640be320 0x7f67640c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6763fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f6764155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f67640be320 0x7f67640c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34504/0 (socket says 172.21.15.160:34504) 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 -- 172.21.15.160:0/1223128856 learned_addr learned my addr 172.21.15.160:0/1223128856 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 -- 172.21.15.160:0/1223128856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 msgr2=0x7f6764155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 --2- 172.21.15.160:0/1223128856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f6764155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 -- 172.21.15.160:0/1223128856 >> v1:172.21.15.188:6789/0 conn(0x7f67640c5610 legacy=0x7f6764155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:48.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.952+0000 7f6762ffd640 1 -- 172.21.15.160:0/1223128856 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67640c77d0 con 0x7f67640be320 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6762ffd640 1 --2- 172.21.15.160:0/1223128856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640be320 0x7f67640c59f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f674c009870 tx=0x7f674c02a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d9840bcf725dad4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f67627fc640 1 -- 172.21.15.160:0/1223128856 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c037070 con 0x7f67640be320 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f67627fc640 1 -- 172.21.15.160:0/1223128856 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c032440 con 0x7f67640be320 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f67627fc640 1 -- 172.21.15.160:0/1223128856 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c032810 con 0x7f67640be320 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 -- 172.21.15.160:0/1223128856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640be320 msgr2=0x7f67640c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 --2- 172.21.15.160:0/1223128856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640be320 0x7f67640c59f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f674c009870 tx=0x7f674c02a130 comp rx=0 tx=0).stop 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 -- 172.21.15.160:0/1223128856 shutdown_connections 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 --2- 172.21.15.160:0/1223128856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640be320 0x7f67640c59f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 --2- 172.21.15.160:0/1223128856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f6764155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 -- 172.21.15.160:0/1223128856 >> 172.21.15.160:0/1223128856 conn(0x7f6764072e90 msgr2=0x7f67640ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:48.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 -- 172.21.15.160:0/1223128856 shutdown_connections 2024-05-09T07:17:48.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.953+0000 7f6769d90640 1 -- 172.21.15.160:0/1223128856 wait complete. 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 Processor -- start 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 -- start start 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67640be320 0x7f67641ee860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 0x7f67640c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f67640c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f67640ad6a0 con 0x7f67640c5610 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f67640c7d90 con 0x7f67640be320 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.955+0000 7f6769d90640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f6764155f60 con 0x7f67640c8b90 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 0x7f67640c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 0x7f67640c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34510/0 (socket says 172.21.15.160:34510) 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 -- 172.21.15.160:0/1124152931 learned_addr learned my addr 172.21.15.160:0/1124152931 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:48.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f67637fe640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67640be320 0x7f67641ee860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6763fff640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f67640c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 -- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 msgr2=0x7f67640c31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f67640c31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 -- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67640be320 msgr2=0x7f67641ee860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67640be320 0x7f67641ee860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 -- 172.21.15.160:0/1124152931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67641ef370 con 0x7f67640c5610 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6762ffd640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 0x7f67640c2c60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f674c000c00 tx=0x7f674c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c041020 con 0x7f67640c5610 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f674c030640 con 0x7f67640c5610 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6764158730 con 0x7f67640c5610 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c037040 con 0x7f67640c5610 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.956+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67640ac530 con 0x7f67640c5610 2024-05-09T07:17:48.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.957+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6764158c50 con 0x7f67640c5610 2024-05-09T07:17:48.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.958+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f674c044020 con 0x7f67640c5610 2024-05-09T07:17:48.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.958+0000 7f675bfff640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f674406be80 0x7f674406e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:48.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.959+0000 7f67637fe640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f674406be80 0x7f674406e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:48.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.959+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f674c0ab860 con 0x7f67640c5610 2024-05-09T07:17:48.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.959+0000 7f67637fe640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f674406be80 0x7f674406e340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f675c001f10 tx=0x7f675c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:48.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 373919 15.62 MB/sec execute 21 sec latency 91.395 ms 2024-05-09T07:17:48.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:48.963+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6764158c50 con 0x7f67640c5610 2024-05-09T07:17:49.185 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.181+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f67640756c0 con 0x7f67640c5610 2024-05-09T07:17:49.186 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.183+0000 7f675bfff640 1 -- 172.21.15.160:0/1124152931 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7f674c04b030 con 0x7f67640c5610 2024-05-09T07:17:49.186 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:17:49.187 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":27,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:14:26.582005+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:17:49.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f674406be80 msgr2=0x7f674406e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.188 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f674406be80 0x7f674406e340 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f675c001f10 tx=0x7f675c009040 comp rx=0 tx=0).stop 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 msgr2=0x7f67640c2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 0x7f67640c2c60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f674c000c00 tx=0x7f674c0046c0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 shutdown_connections 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f674406be80 0x7f674406e340 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f67640c8b90 0x7f67640c31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67640c5610 0x7f67640c2c60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 --2- 172.21.15.160:0/1124152931 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f67640be320 0x7f67641ee860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 >> 172.21.15.160:0/1124152931 conn(0x7f6764072e90 msgr2=0x7f676414f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 shutdown_connections 2024-05-09T07:17:49.189 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.185+0000 7f6769d90640 1 -- 172.21.15.160:0/1124152931 wait complete. 2024-05-09T07:17:49.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:17:49.201 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:17:49.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.308+0000 7f2953431640 1 Processor -- start 2024-05-09T07:17:49.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.308+0000 7f2953431640 1 -- start start 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2953431640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c152a80 0x7f294c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2953431640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f294c0c7ae0 con 0x7f294c0c5690 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2953431640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f294c0c7dc0 con 0x7f294c155e70 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2953431640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f294c152e60 con 0x7f294c152a80 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f29519ce640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f294c0c5690 0x7f294c15fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39454/0 (socket says 172.21.15.160:39454) 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f29519ce640 1 -- 172.21.15.160:0/1936611263 learned_addr learned my addr 172.21.15.160:0/1936611263 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f29511cd640 1 --2- 172.21.15.160:0/1936611263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c152a80 0x7f294c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2769145323 0 0) 0x7f294c0c7ae0 con 0x7f294c0c5690 2024-05-09T07:17:49.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2938003520 con 0x7f294c0c5690 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 577750781 0 0) 0x7f294c0c7dc0 con 0x7f294c155e70 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f294c0c7ae0 con 0x7f294c155e70 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f29511cd640 1 -- 172.21.15.160:0/1936611263 >> v1:172.21.15.188:6789/0 conn(0x7f294c155e70 legacy=0x7f294c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f29511cd640 1 -- 172.21.15.160:0/1936611263 >> v1:172.21.15.160:6789/0 conn(0x7f294c0c5690 legacy=0x7f294c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.309+0000 7f29511cd640 1 -- 172.21.15.160:0/1936611263 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c158b10 con 0x7f294c152a80 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.310+0000 7f29511cd640 1 --2- 172.21.15.160:0/1936611263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c152a80 0x7f294c1568e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f29480097d0 tx=0x7f294802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d74a28962e382c54 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.317 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.310+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948037070 con 0x7f294c152a80 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.310+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294802ae50 con 0x7f294c152a80 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.310+0000 7f2943fff640 1 -- 172.21.15.160:0/1936611263 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948032660 con 0x7f294c152a80 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 -- 172.21.15.160:0/1936611263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c152a80 msgr2=0x7f294c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 --2- 172.21.15.160:0/1936611263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c152a80 0x7f294c1568e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f29480097d0 tx=0x7f294802a100 comp rx=0 tx=0).stop 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 -- 172.21.15.160:0/1936611263 shutdown_connections 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 --2- 172.21.15.160:0/1936611263 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c152a80 0x7f294c1568e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 -- 172.21.15.160:0/1936611263 >> 172.21.15.160:0/1936611263 conn(0x7f294c072ec0 msgr2=0x7f294c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 -- 172.21.15.160:0/1936611263 shutdown_connections 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.311+0000 7f2953431640 1 -- 172.21.15.160:0/1936611263 wait complete. 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.312+0000 7f2953431640 1 Processor -- start 2024-05-09T07:17:49.318 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.312+0000 7f2953431640 1 -- start start 2024-05-09T07:17:49.320 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f2953431640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 0x7f294c0caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f2953431640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f294c152a80 0x7f294c14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f2953431640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c155e70 0x7f294c0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f2953431640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f294c0ad6a0 con 0x7f294c0c5690 2024-05-09T07:17:49.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f2953431640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f294c0c80a0 con 0x7f294c152a80 2024-05-09T07:17:49.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f2953431640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f294c1562f0 con 0x7f294c155e70 2024-05-09T07:17:49.321 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 0x7f294c0caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 0x7f294c0caf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34526/0 (socket says 172.21.15.160:34526) 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 -- 172.21.15.160:0/2074172062 learned_addr learned my addr 172.21.15.160:0/2074172062 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29519ce640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c155e70 0x7f294c0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29509cc640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f294c152a80 0x7f294c14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 -- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c155e70 msgr2=0x7f294c0c9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c155e70 0x7f294c0c9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 -- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f294c152a80 msgr2=0x7f294c14b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f294c152a80 0x7f294c14b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 -- 172.21.15.160:0/2074172062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f294c0c2820 con 0x7f294c0c5690 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.313+0000 7f29511cd640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 0x7f294c0caf50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2948035040 tx=0x7f29480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.314+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948041070 con 0x7f294c0c5690 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.314+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f294803c680 con 0x7f294c0c5690 2024-05-09T07:17:49.322 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.314+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948037040 con 0x7f294c0c5690 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.314+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f294c0cb680 con 0x7f294c0c5690 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.314+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f294c0c2da0 con 0x7f294c0c5690 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.315+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f294c075650 con 0x7f294c0c5690 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.315+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f294803a070 con 0x7f294c0c5690 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.316+0000 7f2941ffb640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f293006be30 0x7f293006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.316+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f29480ab330 con 0x7f294c0c5690 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.316+0000 7f29509cc640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f293006be30 0x7f293006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.316+0000 7f29509cc640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f293006be30 0x7f293006e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f293c007fd0 tx=0x7f293c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.323 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.320+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2948079450 con 0x7f294c0c5690 2024-05-09T07:17:49.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.510+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f294c0ad3a0 con 0x7f294c0c5690 2024-05-09T07:17:49.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.510+0000 7f2941ffb640 1 -- 172.21.15.160:0/2074172062 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f294807d750 con 0x7f294c0c5690 2024-05-09T07:17:49.514 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f293006be30 msgr2=0x7f293006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f293006be30 0x7f293006e2f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f293c007fd0 tx=0x7f293c009040 comp rx=0 tx=0).stop 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 msgr2=0x7f294c0caf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 0x7f294c0caf50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2948035040 tx=0x7f29480046c0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 shutdown_connections 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f293006be30 0x7f293006e2f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f294c152a80 0x7f294c14b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f294c155e70 0x7f294c0c9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 --2- 172.21.15.160:0/2074172062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f294c0c5690 0x7f294c0caf50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 >> 172.21.15.160:0/2074172062 conn(0x7f294c072ec0 msgr2=0x7f294c158110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:49.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 shutdown_connections 2024-05-09T07:17:49.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.513+0000 7f2953431640 1 -- 172.21.15.160:0/2074172062 wait complete. 2024-05-09T07:17:49.528 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:17:49.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.631+0000 7fdca13ce640 1 Processor -- start 2024-05-09T07:17:49.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdca13ce640 1 -- start start 2024-05-09T07:17:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdca13ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdc9c157620 0x7fdc9c072940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdca13ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fdc9c06bb70 0x7fdc9c0c3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdca13ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc9c0c6df0 con 0x7fdc9c072e80 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdca13ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fdc9c0c70d0 con 0x7fdc9c157620 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdca13ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fdc9c0c73b0 con 0x7fdc9c06bb70 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdc9affd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdc9c157620 0x7fdc9c072940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdc9b7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fdc9c072e80 0x7fdc9c0c59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39468/0 (socket says 172.21.15.160:39468) 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdc9affd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdc9c157620 0x7fdc9c072940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34532/0 (socket says 172.21.15.160:34532) 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.632+0000 7fdc9b7fe640 1 -- 172.21.15.160:0/4126845006 learned_addr learned my addr 172.21.15.160:0/4126845006 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc9a7fc640 1 --2- 172.21.15.160:0/4126845006 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c06bb70 0x7fdc9c0c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc9affd640 1 -- 172.21.15.160:0/4126845006 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c06bb70 msgr2=0x7fdc9c0c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc9affd640 1 --2- 172.21.15.160:0/4126845006 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c06bb70 0x7fdc9c0c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc9affd640 1 -- 172.21.15.160:0/4126845006 >> v1:172.21.15.160:6789/0 conn(0x7fdc9c072e80 legacy=0x7fdc9c0c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc9affd640 1 -- 172.21.15.160:0/4126845006 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c0c7690 con 0x7fdc9c157620 2024-05-09T07:17:49.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc99ffb640 1 -- 172.21.15.160:0/4126845006 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 656149494 0 0) 0x7fdc9c0c6df0 con 0x7fdc9c072e80 2024-05-09T07:17:49.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.633+0000 7fdc9affd640 1 --2- 172.21.15.160:0/4126845006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c072940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0c8ac0 tx=0x7fdc8c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=100a3c165fb5341c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdc99ffb640 1 -- 172.21.15.160:0/4126845006 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c037070 con 0x7fdc9c157620 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdc99ffb640 1 -- 172.21.15.160:0/4126845006 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc8c032440 con 0x7fdc9c157620 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdc99ffb640 1 -- 172.21.15.160:0/4126845006 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8c032810 con 0x7fdc9c157620 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 -- 172.21.15.160:0/4126845006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 msgr2=0x7fdc9c072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 --2- 172.21.15.160:0/4126845006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c072940 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdc9c0c8ac0 tx=0x7fdc8c02a1a0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 -- 172.21.15.160:0/4126845006 shutdown_connections 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 --2- 172.21.15.160:0/4126845006 >> v2:172.21.15.188:3300/0 conn(0x7fdc9c06bb70 0x7fdc9c0c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 --2- 172.21.15.160:0/4126845006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c072940 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 -- 172.21.15.160:0/4126845006 >> 172.21.15.160:0/4126845006 conn(0x7fdc9c0a9870 msgr2=0x7fdc9c0a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:49.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 -- 172.21.15.160:0/4126845006 shutdown_connections 2024-05-09T07:17:49.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.634+0000 7fdca13ce640 1 -- 172.21.15.160:0/4126845006 wait complete. 2024-05-09T07:17:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 Processor -- start 2024-05-09T07:17:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 -- start start 2024-05-09T07:17:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9c06bb70 0x7fdc9c0c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c072e80 0x7fdc9c14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c0c2640 con 0x7fdc9c157620 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdc9c0c24c0 con 0x7fdc9c072e80 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdca13ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdc9c0aa520 con 0x7fdc9c06bb70 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdc9b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdc9b7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34548/0 (socket says 172.21.15.160:34548) 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdc9b7fe640 1 -- 172.21.15.160:0/2926486040 learned_addr learned my addr 172.21.15.160:0/2926486040 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdc9a7fc640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c072e80 0x7fdc9c14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.636+0000 7fdc9affd640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9c06bb70 0x7fdc9c0c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc9b7fe640 1 -- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9c06bb70 msgr2=0x7fdc9c0c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc9b7fe640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9c06bb70 0x7fdc9c0c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc9b7fe640 1 -- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c072e80 msgr2=0x7fdc9c14fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc9b7fe640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c072e80 0x7fdc9c14fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc9b7fe640 1 -- 172.21.15.160:0/2926486040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9c0ac6d0 con 0x7fdc9c157620 2024-05-09T07:17:49.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc9b7fe640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c1503d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c7a0 tx=0x7fdc9400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc94018070 con 0x7fdc9c157620 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc9400a640 con 0x7fdc9c157620 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc9c0a6f80 con 0x7fdc9c157620 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9400aab0 con 0x7fdc9c157620 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.637+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc9c0c16a0 con 0x7fdc9c157620 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.638+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc9c071f90 con 0x7fdc9c157620 2024-05-09T07:17:49.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.639+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdc9401b070 con 0x7fdc9c157620 2024-05-09T07:17:49.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.639+0000 7fdc837fe640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdc8806be30 0x7fdc8806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.639+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fdc9408a750 con 0x7fdc9c157620 2024-05-09T07:17:49.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.639+0000 7fdc9affd640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdc8806be30 0x7fdc8806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.640+0000 7fdc9affd640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdc8806be30 0x7fdc8806e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c034000 tx=0x7fdc8c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.644+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdc9c071f90 con 0x7fdc9c157620 2024-05-09T07:17:49.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.834+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdc9c075650 con 0x7fdc9c157620 2024-05-09T07:17:49.838 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.835+0000 7fdc837fe640 1 -- 172.21.15.160:0/2926486040 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fdc94058750 con 0x7fdc9c157620 2024-05-09T07:17:49.838 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdc8806be30 msgr2=0x7fdc8806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdc8806be30 0x7fdc8806e2f0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c034000 tx=0x7fdc8c02e000 comp rx=0 tx=0).stop 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 msgr2=0x7fdc9c1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c1503d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdc9400c7a0 tx=0x7fdc9400cc60 comp rx=0 tx=0).stop 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 shutdown_connections 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdc8806be30 0x7fdc8806e2f0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdc9c072e80 0x7fdc9c14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdc9c157620 0x7fdc9c1503d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 --2- 172.21.15.160:0/2926486040 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdc9c06bb70 0x7fdc9c0c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 >> 172.21.15.160:0/2926486040 conn(0x7fdc9c0a9870 msgr2=0x7fdc9c0a8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:49.841 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 shutdown_connections 2024-05-09T07:17:49.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.837+0000 7fdca13ce640 1 -- 172.21.15.160:0/2926486040 wait complete. 2024-05-09T07:17:49.854 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:17:49.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 Processor -- start 2024-05-09T07:17:49.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 -- start start 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f71dc152a80 0x7f71dc1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71dc0c5690 0x7f71dc0c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71dc0c7ae0 con 0x7f71dc155e70 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f71dc152e60 con 0x7f71dc152a80 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e2ca5640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f71dc0c80a0 con 0x7f71dc0c5690 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.956+0000 7f71e0a41640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f71dc152a80 0x7f71dc1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71d3fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f71dc155e70 0x7f71dc159310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39470/0 (socket says 172.21.15.160:39470) 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71d3fff640 1 -- 172.21.15.160:0/2888121015 learned_addr learned my addr 172.21.15.160:0/2888121015 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71e1242640 1 --2- 172.21.15.160:0/2888121015 >> v2:172.21.15.188:3300/0 conn(0x7f71dc0c5690 0x7f71dc0c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71e0a41640 1 -- 172.21.15.160:0/2888121015 >> v2:172.21.15.188:3300/0 conn(0x7f71dc0c5690 msgr2=0x7f71dc0c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71e0a41640 1 --2- 172.21.15.160:0/2888121015 >> v2:172.21.15.188:3300/0 conn(0x7f71dc0c5690 0x7f71dc0c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71e0a41640 1 -- 172.21.15.160:0/2888121015 >> v1:172.21.15.160:6789/0 conn(0x7f71dc155e70 legacy=0x7f71dc159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71e0a41640 1 -- 172.21.15.160:0/2888121015 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0c7dc0 con 0x7f71dc152a80 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.957+0000 7f71d37fe640 1 -- 172.21.15.160:0/2888121015 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2165873216 0 0) 0x7f71dc0c7ae0 con 0x7f71dc155e70 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e0a41640 1 --2- 172.21.15.160:0/2888121015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc152a80 0x7f71dc1568e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b0e80 tx=0x7f71d802a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db0c0f25fd404429 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71d37fe640 1 -- 172.21.15.160:0/2888121015 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8007740 con 0x7f71dc152a80 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71d37fe640 1 -- 172.21.15.160:0/2888121015 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d80078e0 con 0x7f71dc152a80 2024-05-09T07:17:49.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71d37fe640 1 -- 172.21.15.160:0/2888121015 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8007cb0 con 0x7f71dc152a80 2024-05-09T07:17:49.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 -- 172.21.15.160:0/2888121015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc152a80 msgr2=0x7f71dc1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/2888121015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc152a80 0x7f71dc1568e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f71dc0b0e80 tx=0x7f71d802a560 comp rx=0 tx=0).stop 2024-05-09T07:17:49.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 -- 172.21.15.160:0/2888121015 shutdown_connections 2024-05-09T07:17:49.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/2888121015 >> v2:172.21.15.188:3300/0 conn(0x7f71dc0c5690 0x7f71dc0c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/2888121015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc152a80 0x7f71dc1568e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 -- 172.21.15.160:0/2888121015 >> 172.21.15.160:0/2888121015 conn(0x7f71dc072ec0 msgr2=0x7f71dc0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 -- 172.21.15.160:0/2888121015 shutdown_connections 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.958+0000 7f71e2ca5640 1 -- 172.21.15.160:0/2888121015 wait complete. 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.959+0000 7f71e2ca5640 1 Processor -- start 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 -- start start 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 0x7f71dc0c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71dc152a80 0x7f71dc0ab930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc155e70 0x7f71dc1eb610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc156320 con 0x7f71dc0c5690 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f71dc158df0 con 0x7f71dc155e70 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e2ca5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71dc0ad6a0 con 0x7f71dc152a80 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e0a41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 0x7f71dc0c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e0a41640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 0x7f71dc0c0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34566/0 (socket says 172.21.15.160:34566) 2024-05-09T07:17:49.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.960+0000 7f71e0a41640 1 -- 172.21.15.160:0/1598177697 learned_addr learned my addr 172.21.15.160:0/1598177697 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71d3fff640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71dc152a80 0x7f71dc0ab930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e1242640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc155e70 0x7f71dc1eb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e0a41640 1 -- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71dc152a80 msgr2=0x7f71dc0ab930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e0a41640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71dc152a80 0x7f71dc0ab930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e0a41640 1 -- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc155e70 msgr2=0x7f71dc1eb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e0a41640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc155e70 0x7f71dc1eb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e0a41640 1 -- 172.21.15.160:0/1598177697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71dc0c9880 con 0x7f71dc0c5690 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e0a41640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 0x7f71dc0c0a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009e80 tx=0x7f71d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d803f070 con 0x7f71dc0c5690 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71d8031c40 con 0x7f71dc0c5690 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71d8009450 con 0x7f71dc0c5690 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71dc151450 con 0x7f71dc0c5690 2024-05-09T07:17:49.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.961+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71dc0c0f80 con 0x7f71dc0c5690 2024-05-09T07:17:49.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 377236 15.65 MB/sec execute 22 sec latency 22.980 ms 2024-05-09T07:17:49.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.962+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71dc075650 con 0x7f71dc0c5690 2024-05-09T07:17:49.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.963+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f71d8043410 con 0x7f71dc0c5690 2024-05-09T07:17:49.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.963+0000 7f71d17fa640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:49.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.963+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f71d80ac470 con 0x7f71dc0c5690 2024-05-09T07:17:49.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.963+0000 7f71d3fff640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:49.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.964+0000 7f71d3fff640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f71c806be60 0x7f71c806e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f71dc145ca0 tx=0x7f71cc007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:49.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:49.968+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f71d807a470 con 0x7f71dc0c5690 2024-05-09T07:17:50.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.171+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f71dc0ad3a0 con 0x7f71dc0c5690 2024-05-09T07:17:50.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.172+0000 7f71d17fa640 1 -- 172.21.15.160:0/1598177697 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f71d807e770 con 0x7f71dc0c5690 2024-05-09T07:17:50.176 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f71c806be60 msgr2=0x7f71c806e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f71c806be60 0x7f71c806e320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f71dc145ca0 tx=0x7f71cc007d50 comp rx=0 tx=0).stop 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 msgr2=0x7f71dc0c0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 0x7f71dc0c0a40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f71d8009e80 tx=0x7f71d8004a40 comp rx=0 tx=0).stop 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 shutdown_connections 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f71c806be60 0x7f71c806e320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f71dc155e70 0x7f71dc1eb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71dc152a80 0x7f71dc0ab930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 --2- 172.21.15.160:0/1598177697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f71dc0c5690 0x7f71dc0c0a40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 >> 172.21.15.160:0/1598177697 conn(0x7f71dc072ec0 msgr2=0x7f71dc14ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:50.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 shutdown_connections 2024-05-09T07:17:50.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.174+0000 7f71e2ca5640 1 -- 172.21.15.160:0/1598177697 wait complete. 2024-05-09T07:17:50.189 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:17:50.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.290+0000 7f070be15640 1 Processor -- start 2024-05-09T07:17:50.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.291+0000 7f070be15640 1 -- start start 2024-05-09T07:17:50.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.291+0000 7f070be15640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f07040c2bd0 0x7f07041526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.291+0000 7f070be15640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f07040c2030 0x7f070414ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.291+0000 7f070be15640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07040c3090 con 0x7f070406cd20 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.291+0000 7f070be15640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f07040c1550 con 0x7f07040c2030 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.291+0000 7f070be15640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f07040c1830 con 0x7f07040c2bd0 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f07040c2030 0x7f070414ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f07040c2030 0x7f070414ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34578/0 (socket says 172.21.15.160:34578) 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 -- 172.21.15.160:0/3006254585 learned_addr learned my addr 172.21.15.160:0/3006254585 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f07093b0640 1 --2- 172.21.15.160:0/3006254585 >> v2:172.21.15.188:3300/0 conn(0x7f07040c2bd0 0x7f07041526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 -- 172.21.15.160:0/3006254585 >> v2:172.21.15.188:3300/0 conn(0x7f07040c2bd0 msgr2=0x7f07041526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 --2- 172.21.15.160:0/3006254585 >> v2:172.21.15.188:3300/0 conn(0x7f07040c2bd0 0x7f07041526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 -- 172.21.15.160:0/3006254585 >> v1:172.21.15.188:6789/0 conn(0x7f070406cd20 legacy=0x7f07040c48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.292+0000 7f070a3b2640 1 -- 172.21.15.160:0/3006254585 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07040c1b10 con 0x7f07040c2030 2024-05-09T07:17:50.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070a3b2640 1 --2- 172.21.15.160:0/3006254585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07040c2030 0x7f070414ece0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f070406e6b0 tx=0x7f06fc02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa07fd673d06687 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f0708baf640 1 -- 172.21.15.160:0/3006254585 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc036070 con 0x7f07040c2030 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f0708baf640 1 -- 172.21.15.160:0/3006254585 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06fc031600 con 0x7f07040c2030 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f0708baf640 1 -- 172.21.15.160:0/3006254585 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0319d0 con 0x7f07040c2030 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 -- 172.21.15.160:0/3006254585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07040c2030 msgr2=0x7f070414ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 --2- 172.21.15.160:0/3006254585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07040c2030 0x7f070414ece0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f070406e6b0 tx=0x7f06fc02a760 comp rx=0 tx=0).stop 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 -- 172.21.15.160:0/3006254585 shutdown_connections 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 --2- 172.21.15.160:0/3006254585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f07040c2030 0x7f070414ece0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 --2- 172.21.15.160:0/3006254585 >> v2:172.21.15.188:3300/0 conn(0x7f07040c2bd0 0x7f07041526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 -- 172.21.15.160:0/3006254585 >> 172.21.15.160:0/3006254585 conn(0x7f07040c69b0 msgr2=0x7f07040c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 -- 172.21.15.160:0/3006254585 shutdown_connections 2024-05-09T07:17:50.298 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.293+0000 7f070be15640 1 -- 172.21.15.160:0/3006254585 wait complete. 2024-05-09T07:17:50.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.295+0000 7f070be15640 1 Processor -- start 2024-05-09T07:17:50.299 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.295+0000 7f070be15640 1 -- start start 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070be15640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 0x7f070413dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070be15640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040c2030 0x7f070413e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070be15640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040c2bd0 0x7f07040715b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070be15640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f070406d100 con 0x7f070406cd20 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070be15640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f070406c700 con 0x7f07040c2bd0 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070be15640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f070406d280 con 0x7f07040c2030 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f0709bb1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 0x7f070413dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f07093b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040c2030 0x7f070413e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f070a3b2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040c2bd0 0x7f07040715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f0709bb1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 0x7f070413dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34584/0 (socket says 172.21.15.160:34584) 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f0709bb1640 1 -- 172.21.15.160:0/3175584132 learned_addr learned my addr 172.21.15.160:0/3175584132 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.296+0000 7f07093b0640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040c2030 0x7f070413e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:41432/0 (socket says 172.21.15.160:41432) 2024-05-09T07:17:50.301 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f0709bb1640 1 -- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040c2030 msgr2=0x7f070413e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f0709bb1640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040c2030 0x7f070413e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f0709bb1640 1 -- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040c2bd0 msgr2=0x7f07040715b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f0709bb1640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040c2bd0 0x7f07040715b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f0709bb1640 1 -- 172.21.15.160:0/3175584132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0704070dd0 con 0x7f070406cd20 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f070a3b2640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040c2bd0 0x7f07040715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f0709bb1640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 0x7f070413dc70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f070000c3f0 tx=0x7f070000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07000052d0 con 0x7f070406cd20 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0700005490 con 0x7f070406cd20 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0700007260 con 0x7f070406cd20 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0704073810 con 0x7f070406cd20 2024-05-09T07:17:50.302 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.297+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07041ec0a0 con 0x7f070406cd20 2024-05-09T07:17:50.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.298+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070414db50 con 0x7f070406cd20 2024-05-09T07:17:50.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.299+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f0700007400 con 0x7f070406cd20 2024-05-09T07:17:50.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.299+0000 7f06fa7fc640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f06e806be30 0x7f06e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.299+0000 7f07093b0640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f06e806be30 0x7f06e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.299+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f070008a600 con 0x7f070406cd20 2024-05-09T07:17:50.303 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.300+0000 7f07093b0640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f06e806be30 0x7f06e806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f070414bb70 tx=0x7f06f4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.304+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f070414db50 con 0x7f070406cd20 2024-05-09T07:17:50.492 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.489+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0704075650 con 0x7f070406cd20 2024-05-09T07:17:50.493 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.489+0000 7f06fa7fc640 1 -- 172.21.15.160:0/3175584132 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f0700058600 con 0x7f070406cd20 2024-05-09T07:17:50.493 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:17:50.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f06e806be30 msgr2=0x7f06e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f06e806be30 0x7f06e806e2f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f070414bb70 tx=0x7f06f4009190 comp rx=0 tx=0).stop 2024-05-09T07:17:50.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 msgr2=0x7f070413dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.495 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 0x7f070413dc70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f070000c3f0 tx=0x7f070000c8b0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 shutdown_connections 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f06e806be30 0x7f06e806e2f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f07040c2bd0 0x7f07040715b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f07040c2030 0x7f070413e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 --2- 172.21.15.160:0/3175584132 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f070406cd20 0x7f070413dc70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 >> 172.21.15.160:0/3175584132 conn(0x7f07040c69b0 msgr2=0x7f070414d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.491+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 shutdown_connections 2024-05-09T07:17:50.496 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.492+0000 7f070be15640 1 -- 172.21.15.160:0/3175584132 wait complete. 2024-05-09T07:17:50.506 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T07:17:50.606 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.603+0000 7fc3c4e95640 1 Processor -- start 2024-05-09T07:17:50.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.603+0000 7fc3c4e95640 1 -- start start 2024-05-09T07:17:50.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3c4e95640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc3c00aa470 0x7fc3c006e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3c4e95640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc3c00ac620 0x7fc3c006b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3c4e95640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3c0153300 con 0x7fc3c006e400 2024-05-09T07:17:50.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3c4e95640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c0153f10 con 0x7fc3c00ac620 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3c4e95640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc3c01535e0 con 0x7fc3c00aa470 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc3c00ac620 0x7fc3c006b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc3c00ac620 0x7fc3c006b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34590/0 (socket says 172.21.15.160:34590) 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 -- 172.21.15.160:0/3748052181 learned_addr learned my addr 172.21.15.160:0/3748052181 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3b5d9b640 1 --2- 172.21.15.160:0/3748052181 >> v2:172.21.15.188:3300/0 conn(0x7fc3c00aa470 0x7fc3c006e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 -- 172.21.15.160:0/3748052181 >> v2:172.21.15.188:3300/0 conn(0x7fc3c00aa470 msgr2=0x7fc3c006e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 --2- 172.21.15.160:0/3748052181 >> v2:172.21.15.188:3300/0 conn(0x7fc3c00aa470 0x7fc3c006e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 -- 172.21.15.160:0/3748052181 >> v1:172.21.15.188:6789/0 conn(0x7fc3c006e400 legacy=0x7fc3c006b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bed9d640 1 -- 172.21.15.160:0/3748052181 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c01538c0 con 0x7fc3c00ac620 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.604+0000 7fc3bdd9b640 1 -- 172.21.15.160:0/3748052181 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2003421184 0 0) 0x7fc3c0153300 con 0x7fc3c006e400 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3bed9d640 1 --2- 172.21.15.160:0/3748052181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00ac620 0x7fc3c006b870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00c8220 tx=0x7fc3b802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ba7869f1886b07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3bdd9b640 1 -- 172.21.15.160:0/3748052181 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8036070 con 0x7fc3c00ac620 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3bdd9b640 1 -- 172.21.15.160:0/3748052181 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3b8031600 con 0x7fc3c00ac620 2024-05-09T07:17:50.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3bdd9b640 1 -- 172.21.15.160:0/3748052181 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80319d0 con 0x7fc3c00ac620 2024-05-09T07:17:50.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3748052181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00ac620 msgr2=0x7fc3c006b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3748052181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00ac620 0x7fc3c006b870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc3c00c8220 tx=0x7fc3b802a610 comp rx=0 tx=0).stop 2024-05-09T07:17:50.612 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3748052181 shutdown_connections 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3748052181 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00ac620 0x7fc3c006b870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3748052181 >> v2:172.21.15.188:3300/0 conn(0x7fc3c00aa470 0x7fc3c006e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3748052181 >> 172.21.15.160:0/3748052181 conn(0x7fc3c00c76e0 msgr2=0x7fc3c00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3748052181 shutdown_connections 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.605+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3748052181 wait complete. 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.607+0000 7fc3c4e95640 1 Processor -- start 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.607+0000 7fc3c4e95640 1 -- start start 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c006e400 0x7fc3c00c0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 0x7fc3c00728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c00ac620 0x7fc3c0143670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c0150bc0 con 0x7fc3c00aa470 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3c006bdb0 con 0x7fc3c00ac620 2024-05-09T07:17:50.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3c00acd60 con 0x7fc3c006e400 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 0x7fc3c00728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 0x7fc3c00728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34594/0 (socket says 172.21.15.160:34594) 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 -- 172.21.15.160:0/3305116366 learned_addr learned my addr 172.21.15.160:0/3305116366 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3be59c640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c006e400 0x7fc3c00c0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3bed9d640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c00ac620 0x7fc3c0143670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 -- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c006e400 msgr2=0x7fc3c00c0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c006e400 0x7fc3c00c0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 -- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c00ac620 msgr2=0x7fc3c0143670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c00ac620 0x7fc3c0143670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 -- 172.21.15.160:0/3305116366 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3c01eac90 con 0x7fc3c00aa470 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b5d9b640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 0x7fc3c00728c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800e9b0 tx=0x7fc3a800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8017070 con 0x7fc3c00aa470 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3a800c9a0 con 0x7fc3c00aa470 2024-05-09T07:17:50.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3c01eaf70 con 0x7fc3c00aa470 2024-05-09T07:17:50.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3a800cd70 con 0x7fc3c00aa470 2024-05-09T07:17:50.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.608+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3c01eb530 con 0x7fc3c00aa470 2024-05-09T07:17:50.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.610+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3c00756c0 con 0x7fc3c00aa470 2024-05-09T07:17:50.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.610+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc3a801a070 con 0x7fc3c00aa470 2024-05-09T07:17:50.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.610+0000 7fc3b77fe640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc3a006be30 0x7fc3a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.611+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc3a808b4d0 con 0x7fc3c00aa470 2024-05-09T07:17:50.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.611+0000 7fc3be59c640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc3a006be30 0x7fc3a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.611+0000 7fc3be59c640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc3a006be30 0x7fc3a006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0000f60 tx=0x7fc3b0009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.618 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.615+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc3a80594d0 con 0x7fc3c00aa470 2024-05-09T07:17:50.809 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.806+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc3c00ad440 con 0x7fc3c00aa470 2024-05-09T07:17:50.810 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.806+0000 7fc3b77fe640 1 -- 172.21.15.160:0/3305116366 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fc3a805d7d0 con 0x7fc3c00aa470 2024-05-09T07:17:50.810 INFO:teuthology.orchestra.run.smithi160.stderr:listed 0 entries 2024-05-09T07:17:50.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc3a006be30 msgr2=0x7fc3a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc3a006be30 0x7fc3a006e2f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc3b0000f60 tx=0x7fc3b0009040 comp rx=0 tx=0).stop 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 msgr2=0x7fc3c00728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 0x7fc3c00728c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc3a800e9b0 tx=0x7fc3a800ee70 comp rx=0 tx=0).stop 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 shutdown_connections 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc3a006be30 0x7fc3a006e2f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3c00ac620 0x7fc3c0143670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3c00aa470 0x7fc3c00728c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 --2- 172.21.15.160:0/3305116366 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3c006e400 0x7fc3c00c0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 >> 172.21.15.160:0/3305116366 conn(0x7fc3c00c76e0 msgr2=0x7fc3c006c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 shutdown_connections 2024-05-09T07:17:50.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.809+0000 7fc3c4e95640 1 -- 172.21.15.160:0/3305116366 wait complete. 2024-05-09T07:17:50.824 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2024-05-09T07:17:50.824 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:17:50.840 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:17:50.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 380733 15.80 MB/sec execute 23 sec latency 21.076 ms 2024-05-09T07:17:50.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.978+0000 7fc3a6e9a640 1 Processor -- start 2024-05-09T07:17:50.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.978+0000 7fc3a6e9a640 1 -- start start 2024-05-09T07:17:50.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a6e9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a0153300 0x7fc3a00c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a6e9a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc3a00c5e20 0x7fc3a00bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a6e9a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc3a01563b0 con 0x7fc3a00a5fb0 2024-05-09T07:17:50.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a6e9a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc3a00c1630 con 0x7fc3a00c5e20 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a6e9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a00c19d0 con 0x7fc3a0153300 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc3a00c5e20 0x7fc3a00bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc3a00c5e20 0x7fc3a00bc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34596/0 (socket says 172.21.15.160:34596) 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 -- 172.21.15.160:0/326589533 learned_addr learned my addr 172.21.15.160:0/326589533 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a4c36640 1 --2- 172.21.15.160:0/326589533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a0153300 0x7fc3a00c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 -- 172.21.15.160:0/326589533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a0153300 msgr2=0x7fc3a00c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 --2- 172.21.15.160:0/326589533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a0153300 0x7fc3a00c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 -- 172.21.15.160:0/326589533 >> v1:172.21.15.188:6789/0 conn(0x7fc3a00a5fb0 legacy=0x7fc3a00bbe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.979+0000 7fc3a5437640 1 -- 172.21.15.160:0/326589533 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a0156690 con 0x7fc3a00c5e20 2024-05-09T07:17:50.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.980+0000 7fc3a5437640 1 --2- 172.21.15.160:0/326589533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a00c5e20 0x7fc3a00bc530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc39802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adff32a2e2b0cbc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.980+0000 7fc39f7fe640 1 -- 172.21.15.160:0/326589533 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398037070 con 0x7fc3a00c5e20 2024-05-09T07:17:50.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.980+0000 7fc39f7fe640 1 -- 172.21.15.160:0/326589533 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc398032440 con 0x7fc3a00c5e20 2024-05-09T07:17:50.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.980+0000 7fc39f7fe640 1 -- 172.21.15.160:0/326589533 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398032810 con 0x7fc3a00c5e20 2024-05-09T07:17:50.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/326589533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a00c5e20 msgr2=0x7fc3a00bc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/326589533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a00c5e20 0x7fc3a00bc530 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc39802a270 comp rx=0 tx=0).stop 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/326589533 shutdown_connections 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/326589533 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a00c5e20 0x7fc3a00bc530 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/326589533 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a0153300 0x7fc3a00c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/326589533 >> 172.21.15.160:0/326589533 conn(0x7fc3a00c7580 msgr2=0x7fc3a00a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/326589533 shutdown_connections 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.981+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/326589533 wait complete. 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.982+0000 7fc3a6e9a640 1 Processor -- start 2024-05-09T07:17:50.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.982+0000 7fc3a6e9a640 1 -- start start 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a6e9a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3a00a5fb0 0x7fc3a0154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a6e9a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a00c5e20 0x7fc3a0073000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a6e9a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 0x7fc3a014df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a6e9a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00c2420 con 0x7fc3a0153300 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a6e9a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc3a00c1f00 con 0x7fc3a00a5fb0 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a6e9a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc3a00bd220 con 0x7fc3a00c5e20 2024-05-09T07:17:50.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 0x7fc3a014df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 0x7fc3a014df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34598/0 (socket says 172.21.15.160:34598) 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 -- 172.21.15.160:0/3714107180 learned_addr learned my addr 172.21.15.160:0/3714107180 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a4c36640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3a00a5fb0 0x7fc3a0154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc39ffff640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a00c5e20 0x7fc3a0073000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 -- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a00c5e20 msgr2=0x7fc3a0073000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a00c5e20 0x7fc3a0073000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 -- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3a00a5fb0 msgr2=0x7fc3a0154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3a00a5fb0 0x7fc3a0154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 -- 172.21.15.160:0/3714107180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3a01e99d0 con 0x7fc3a0153300 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.983+0000 7fc3a5437640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 0x7fc3a014df20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc3980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.984+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398041020 con 0x7fc3a0153300 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.984+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc39803d640 con 0x7fc3a0153300 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.984+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc398037040 con 0x7fc3a0153300 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.984+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3a014ecb0 con 0x7fc3a0153300 2024-05-09T07:17:50.989 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.984+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3a014e9d0 con 0x7fc3a0153300 2024-05-09T07:17:50.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.985+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3a01e9ba0 con 0x7fc3a0153300 2024-05-09T07:17:50.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.985+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc398044020 con 0x7fc3a0153300 2024-05-09T07:17:50.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.986+0000 7fc39d7fa640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc38006be60 0x7fc38006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:50.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.986+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc3980ab810 con 0x7fc3a0153300 2024-05-09T07:17:50.990 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.986+0000 7fc3a4c36640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc38006be60 0x7fc38006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:50.991 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.986+0000 7fc3a4c36640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc38006be60 0x7fc38006e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3a006bfa0 tx=0x7fc388009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:50.993 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:50.990+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc39804b030 con 0x7fc3a0153300 2024-05-09T07:17:51.203 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.200+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc3a00abe50 con 0x7fc3a0153300 2024-05-09T07:17:51.204 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.201+0000 7fc39d7fa640 1 -- 172.21.15.160:0/3714107180 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7fc398079790 con 0x7fc3a0153300 2024-05-09T07:17:51.205 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:17:51.205 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":27,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:14:26.582005+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:17:51.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc38006be60 msgr2=0x7fc38006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc38006be60 0x7fc38006e320 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3a006bfa0 tx=0x7fc388009190 comp rx=0 tx=0).stop 2024-05-09T07:17:51.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 msgr2=0x7fc3a014df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 0x7fc3a014df20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc398009870 tx=0x7fc3980046c0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 shutdown_connections 2024-05-09T07:17:51.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc38006be60 0x7fc38006e320 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3a0153300 0x7fc3a014df20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc3a00c5e20 0x7fc3a0073000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 --2- 172.21.15.160:0/3714107180 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc3a00a5fb0 0x7fc3a0154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 >> 172.21.15.160:0/3714107180 conn(0x7fc3a00c7580 msgr2=0x7fc3a006cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:51.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 shutdown_connections 2024-05-09T07:17:51.208 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.203+0000 7fc3a6e9a640 1 -- 172.21.15.160:0/3714107180 wait complete. 2024-05-09T07:17:51.220 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:17:51.220 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:17:51.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 Processor -- start 2024-05-09T07:17:51.327 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 -- start start 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f85d8152a80 0x7f85d81568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f85d8155e70 0x7f85d8159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85d80c7ae0 con 0x7f85d80c5690 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f85d8152e60 con 0x7f85d8152a80 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.324+0000 7f85dda38640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f85d80c80a0 con 0x7f85d8155e70 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f85d8152a80 0x7f85d81568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f85d8152a80 0x7f85d81568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34604/0 (socket says 172.21.15.160:34604) 2024-05-09T07:17:51.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d77fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f85d80c5690 0x7f85d806c520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39484/0 (socket says 172.21.15.160:39484) 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 -- 172.21.15.160:0/757034621 learned_addr learned my addr 172.21.15.160:0/757034621 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d67fc640 1 --2- 172.21.15.160:0/757034621 >> v2:172.21.15.188:3300/0 conn(0x7f85d8155e70 0x7f85d8159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 -- 172.21.15.160:0/757034621 >> v2:172.21.15.188:3300/0 conn(0x7f85d8155e70 msgr2=0x7f85d8159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 --2- 172.21.15.160:0/757034621 >> v2:172.21.15.188:3300/0 conn(0x7f85d8155e70 0x7f85d8159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 -- 172.21.15.160:0/757034621 >> v1:172.21.15.160:6789/0 conn(0x7f85d80c5690 legacy=0x7f85d806c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d6ffd640 1 -- 172.21.15.160:0/757034621 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d80c7dc0 con 0x7f85d8152a80 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.325+0000 7f85d5ffb640 1 -- 172.21.15.160:0/757034621 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 864164022 0 0) 0x7f85d80c7ae0 con 0x7f85d80c5690 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85d6ffd640 1 --2- 172.21.15.160:0/757034621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d81568e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ad340 tx=0x7f85c802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28243671a2547dc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85d5ffb640 1 -- 172.21.15.160:0/757034621 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8037070 con 0x7f85d8152a80 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85d5ffb640 1 -- 172.21.15.160:0/757034621 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85c8032440 con 0x7f85d8152a80 2024-05-09T07:17:51.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85d5ffb640 1 -- 172.21.15.160:0/757034621 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85c8032810 con 0x7f85d8152a80 2024-05-09T07:17:51.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 -- 172.21.15.160:0/757034621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 msgr2=0x7f85d81568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 --2- 172.21.15.160:0/757034621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d81568e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f85d80ad340 tx=0x7f85c802a150 comp rx=0 tx=0).stop 2024-05-09T07:17:51.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 -- 172.21.15.160:0/757034621 shutdown_connections 2024-05-09T07:17:51.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 --2- 172.21.15.160:0/757034621 >> v2:172.21.15.188:3300/0 conn(0x7f85d8155e70 0x7f85d8159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 --2- 172.21.15.160:0/757034621 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d81568e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 -- 172.21.15.160:0/757034621 >> 172.21.15.160:0/757034621 conn(0x7f85d8072ec0 msgr2=0x7f85d80ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:51.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 -- 172.21.15.160:0/757034621 shutdown_connections 2024-05-09T07:17:51.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.326+0000 7f85dda38640 1 -- 172.21.15.160:0/757034621 wait complete. 2024-05-09T07:17:51.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.327+0000 7f85dda38640 1 Processor -- start 2024-05-09T07:17:51.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 -- start start 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d80c5690 0x7f85d8148000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d814a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d8155e70 0x7f85d814a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8156320 con 0x7f85d8152a80 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f85d8158df0 con 0x7f85d80c5690 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85dda38640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f85d80ad6a0 con 0x7f85d8155e70 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85d67fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d814a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85d67fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d814a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34614/0 (socket says 172.21.15.160:34614) 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85d67fc640 1 -- 172.21.15.160:0/4108247824 learned_addr learned my addr 172.21.15.160:0/4108247824 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85d6ffd640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d80c5690 0x7f85d8148000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.328+0000 7f85d77fe640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d8155e70 0x7f85d814a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85d67fc640 1 -- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d8155e70 msgr2=0x7f85d814a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85d67fc640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d8155e70 0x7f85d814a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85d67fc640 1 -- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d80c5690 msgr2=0x7f85d8148000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85d67fc640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d80c5690 0x7f85d8148000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85d67fc640 1 -- 172.21.15.160:0/4108247824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d8148540 con 0x7f85d8152a80 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85d67fc640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d814a0b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00e9b0 tx=0x7f85cc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc017070 con 0x7f85d8152a80 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85cc00c9a0 con 0x7f85d8152a80 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d8147d60 con 0x7f85d8152a80 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc00cd90 con 0x7f85d8152a80 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.329+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d81e1780 con 0x7f85d8152a80 2024-05-09T07:17:51.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.330+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8075650 con 0x7f85d8152a80 2024-05-09T07:17:51.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.330+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f85cc01a070 con 0x7f85d8152a80 2024-05-09T07:17:51.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.331+0000 7f85bffff640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f85c406be30 0x7f85c406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.331+0000 7f85d6ffd640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f85c406be30 0x7f85c406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.331+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f85cc08b450 con 0x7f85d8152a80 2024-05-09T07:17:51.336 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.332+0000 7f85d6ffd640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f85c406be30 0x7f85c406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85c8034000 tx=0x7f85c802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.339 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.335+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f85d8148540 con 0x7f85d8152a80 2024-05-09T07:17:51.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.515+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f85d81582d0 con 0x7f85d8152a80 2024-05-09T07:17:51.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.515+0000 7f85bffff640 1 -- 172.21.15.160:0/4108247824 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f85cc059450 con 0x7f85d8152a80 2024-05-09T07:17:51.519 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:17:51.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.517+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f85c406be30 msgr2=0x7f85c406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.521 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.517+0000 7f85dda38640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f85c406be30 0x7f85c406e2f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f85c8034000 tx=0x7f85c802e000 comp rx=0 tx=0).stop 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 msgr2=0x7f85d814a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d814a0b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00e9b0 tx=0x7f85cc00ee70 comp rx=0 tx=0).stop 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 shutdown_connections 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f85c406be30 0x7f85c406e2f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f85d8155e70 0x7f85d814a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f85d8152a80 0x7f85d814a0b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 --2- 172.21.15.160:0/4108247824 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f85d80c5690 0x7f85d8148000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 >> 172.21.15.160:0/4108247824 conn(0x7f85d8072ec0 msgr2=0x7f85d814c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 shutdown_connections 2024-05-09T07:17:51.522 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.518+0000 7f85dda38640 1 -- 172.21.15.160:0/4108247824 wait complete. 2024-05-09T07:17:51.533 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:17:51.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 Processor -- start 2024-05-09T07:17:51.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 -- start start 2024-05-09T07:17:51.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9590152a80 0x7f95901568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95900c5690 0x7f95900c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f95900c7ae0 con 0x7f9590155e70 2024-05-09T07:17:51.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f9590152e60 con 0x7f9590152a80 2024-05-09T07:17:51.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.632+0000 7f959602a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95900c80a0 con 0x7f95900c5690 2024-05-09T07:17:51.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9590152a80 0x7f95901568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f9590152a80 0x7f95901568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34628/0 (socket says 172.21.15.160:34628) 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 -- 172.21.15.160:0/1425970578 learned_addr learned my addr 172.21.15.160:0/1425970578 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958ffff640 1 --2- 172.21.15.160:0/1425970578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95900c5690 0x7f95900c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 -- 172.21.15.160:0/1425970578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95900c5690 msgr2=0x7f95900c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 --2- 172.21.15.160:0/1425970578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95900c5690 0x7f95900c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 -- 172.21.15.160:0/1425970578 >> v1:172.21.15.188:6789/0 conn(0x7f9590155e70 legacy=0x7f9590159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.633+0000 7f958f7fe640 1 -- 172.21.15.160:0/1425970578 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95900c7dc0 con 0x7f9590152a80 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f958f7fe640 1 --2- 172.21.15.160:0/1425970578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590152a80 0x7f95901568e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9580009870 tx=0x7f958002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4c9b801fbc621 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f958e7fc640 1 -- 172.21.15.160:0/1425970578 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580037070 con 0x7f9590152a80 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f958e7fc640 1 -- 172.21.15.160:0/1425970578 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9580032440 con 0x7f9590152a80 2024-05-09T07:17:51.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f958e7fc640 1 -- 172.21.15.160:0/1425970578 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9580032810 con 0x7f9590152a80 2024-05-09T07:17:51.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 -- 172.21.15.160:0/1425970578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590152a80 msgr2=0x7f95901568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 --2- 172.21.15.160:0/1425970578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590152a80 0x7f95901568e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9580009870 tx=0x7f958002a130 comp rx=0 tx=0).stop 2024-05-09T07:17:51.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 -- 172.21.15.160:0/1425970578 shutdown_connections 2024-05-09T07:17:51.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 --2- 172.21.15.160:0/1425970578 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f95900c5690 0x7f95900c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 --2- 172.21.15.160:0/1425970578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590152a80 0x7f95901568e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 -- 172.21.15.160:0/1425970578 >> 172.21.15.160:0/1425970578 conn(0x7f9590072ec0 msgr2=0x7f95900ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:51.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 -- 172.21.15.160:0/1425970578 shutdown_connections 2024-05-09T07:17:51.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.634+0000 7f959602a640 1 -- 172.21.15.160:0/1425970578 wait complete. 2024-05-09T07:17:51.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.635+0000 7f959602a640 1 Processor -- start 2024-05-09T07:17:51.640 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 -- start start 2024-05-09T07:17:51.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95900c5690 0x7f959014be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9590152a80 0x7f95901e2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 0x7f95901ee300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9590156320 con 0x7f9590155e70 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9590158df0 con 0x7f95900c5690 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f959602a640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f95900ad6a0 con 0x7f9590152a80 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f958ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 0x7f95901ee300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f958ffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 0x7f95901ee300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34640/0 (socket says 172.21.15.160:34640) 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f958ffff640 1 -- 172.21.15.160:0/3214129822 learned_addr learned my addr 172.21.15.160:0/3214129822 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.636+0000 7f958f7fe640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95900c5690 0x7f959014be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958effd640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9590152a80 0x7f95901e2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958ffff640 1 -- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9590152a80 msgr2=0x7f95901e2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958ffff640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9590152a80 0x7f95901e2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958ffff640 1 -- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95900c5690 msgr2=0x7f959014be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958ffff640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95900c5690 0x7f959014be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958ffff640 1 -- 172.21.15.160:0/3214129822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959014e3b0 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f958ffff640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 0x7f95901ee300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f958800efc0 tx=0x7f958800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588018020 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9588014480 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588014870 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95900cb630 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.637+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9590153790 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.638+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9590075650 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.638+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9588005d10 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.639+0000 7f9587fff640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f957006be30 0x7f957006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.639+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f958808bb20 con 0x7f9590155e70 2024-05-09T07:17:51.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.639+0000 7f958f7fe640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f957006be30 0x7f957006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.639+0000 7f958f7fe640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f957006be30 0x7f957006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9580034000 tx=0x7f958002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.646 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.643+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9588022070 con 0x7f9590155e70 2024-05-09T07:17:51.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.820+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95900acb40 con 0x7f9590155e70 2024-05-09T07:17:51.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.820+0000 7f9587fff640 1 -- 172.21.15.160:0/3214129822 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f9588010040 con 0x7f9590155e70 2024-05-09T07:17:51.824 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:17:51.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f957006be30 msgr2=0x7f957006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f957006be30 0x7f957006e2f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9580034000 tx=0x7f958002e000 comp rx=0 tx=0).stop 2024-05-09T07:17:51.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 msgr2=0x7f95901ee300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 0x7f95901ee300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f958800efc0 tx=0x7f958800c490 comp rx=0 tx=0).stop 2024-05-09T07:17:51.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 shutdown_connections 2024-05-09T07:17:51.827 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f957006be30 0x7f957006e2f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9590155e70 0x7f95901ee300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9590152a80 0x7f95901e2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 --2- 172.21.15.160:0/3214129822 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f95900c5690 0x7f959014be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 >> 172.21.15.160:0/3214129822 conn(0x7f9590072ec0 msgr2=0x7f9590151760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:51.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 shutdown_connections 2024-05-09T07:17:51.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.823+0000 7f959602a640 1 -- 172.21.15.160:0/3214129822 wait complete. 2024-05-09T07:17:51.840 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:17:51.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.946+0000 7fe879d4f640 1 Processor -- start 2024-05-09T07:17:51.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.946+0000 7fe879d4f640 1 -- start start 2024-05-09T07:17:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe879d4f640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe8741550e0 0x7fe8740c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe879d4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8740ab310 0x7fe874156280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe879d4f640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8740c20a0 con 0x7fe87406d510 2024-05-09T07:17:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe879d4f640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe87406c500 con 0x7fe8741550e0 2024-05-09T07:17:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe879d4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe87406d080 con 0x7fe8740ab310 2024-05-09T07:17:51.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe873fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe87406d510 0x7fe8740c59c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39490/0 (socket says 172.21.15.160:39490) 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe873fff640 1 -- 172.21.15.160:0/2821100453 learned_addr learned my addr 172.21.15.160:0/2821100453 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe872ffd640 1 --2- 172.21.15.160:0/2821100453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8740ab310 0x7fe874156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.947+0000 7fe8737fe640 1 --2- 172.21.15.160:0/2821100453 >> v2:172.21.15.188:3300/0 conn(0x7fe8741550e0 0x7fe8740c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe8727fc640 1 -- 172.21.15.160:0/2821100453 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1737289214 0 0) 0x7fe8740c20a0 con 0x7fe87406d510 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe8727fc640 1 -- 172.21.15.160:0/2821100453 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe85c003520 con 0x7fe87406d510 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe872ffd640 1 -- 172.21.15.160:0/2821100453 >> v2:172.21.15.188:3300/0 conn(0x7fe8741550e0 msgr2=0x7fe8740c29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe872ffd640 1 --2- 172.21.15.160:0/2821100453 >> v2:172.21.15.188:3300/0 conn(0x7fe8741550e0 0x7fe8740c29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe872ffd640 1 -- 172.21.15.160:0/2821100453 >> v1:172.21.15.160:6789/0 conn(0x7fe87406d510 legacy=0x7fe8740c59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe872ffd640 1 -- 172.21.15.160:0/2821100453 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe87406c750 con 0x7fe8740ab310 2024-05-09T07:17:51.952 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.948+0000 7fe8737fe640 1 --2- 172.21.15.160:0/2821100453 >> v2:172.21.15.188:3300/0 conn(0x7fe8741550e0 0x7fe8740c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:17:51.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe872ffd640 1 --2- 172.21.15.160:0/2821100453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8740ab310 0x7fe874156280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8680097a0 tx=0x7fe86802a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=105b6adf021d53e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe8727fc640 1 -- 172.21.15.160:0/2821100453 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868037070 con 0x7fe8740ab310 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe8727fc640 1 -- 172.21.15.160:0/2821100453 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86802ae40 con 0x7fe8740ab310 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe8727fc640 1 -- 172.21.15.160:0/2821100453 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868032640 con 0x7fe8740ab310 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe879d4f640 1 -- 172.21.15.160:0/2821100453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8740ab310 msgr2=0x7fe874156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe879d4f640 1 --2- 172.21.15.160:0/2821100453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8740ab310 0x7fe874156280 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe8680097a0 tx=0x7fe86802a0f0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe879d4f640 1 -- 172.21.15.160:0/2821100453 shutdown_connections 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe879d4f640 1 --2- 172.21.15.160:0/2821100453 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8740ab310 0x7fe874156280 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe879d4f640 1 --2- 172.21.15.160:0/2821100453 >> v2:172.21.15.188:3300/0 conn(0x7fe8741550e0 0x7fe8740c29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.949+0000 7fe879d4f640 1 -- 172.21.15.160:0/2821100453 >> 172.21.15.160:0/2821100453 conn(0x7fe8740c69b0 msgr2=0x7fe8740c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.950+0000 7fe879d4f640 1 -- 172.21.15.160:0/2821100453 shutdown_connections 2024-05-09T07:17:51.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.950+0000 7fe879d4f640 1 -- 172.21.15.160:0/2821100453 wait complete. 2024-05-09T07:17:51.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.951+0000 7fe879d4f640 1 Processor -- start 2024-05-09T07:17:51.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.951+0000 7fe879d4f640 1 -- start start 2024-05-09T07:17:51.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.951+0000 7fe879d4f640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe87406d510 0x7fe8741e9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 0x7fe874144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8741550e0 0x7fe87414c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe8740c60d0 con 0x7fe8740ab310 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe87406cd10 con 0x7fe87406d510 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe8740c6250 con 0x7fe8741550e0 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 0x7fe874144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 0x7fe874144920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34642/0 (socket says 172.21.15.160:34642) 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe8737fe640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe87406d510 0x7fe8741e9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 -- 172.21.15.160:0/477618733 learned_addr learned my addr 172.21.15.160:0/477618733 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe873fff640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8741550e0 0x7fe87414c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 -- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8741550e0 msgr2=0x7fe87414c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8741550e0 0x7fe87414c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 -- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe87406d510 msgr2=0x7fe8741e9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe87406d510 0x7fe8741e9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 -- 172.21.15.160:0/477618733 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8741eca50 con 0x7fe8740ab310 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe872ffd640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 0x7fe874144920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe868035040 tx=0x7fe8680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868041020 con 0x7fe8740ab310 2024-05-09T07:17:51.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe86803c670 con 0x7fe8740ab310 2024-05-09T07:17:51.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8741ecd30 con 0x7fe8740ab310 2024-05-09T07:17:51.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe868037040 con 0x7fe8740ab310 2024-05-09T07:17:51.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.952+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8741ed2f0 con 0x7fe8740ab310 2024-05-09T07:17:51.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.953+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe874075650 con 0x7fe8740ab310 2024-05-09T07:17:51.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.954+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe868044020 con 0x7fe8740ab310 2024-05-09T07:17:51.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.954+0000 7fe85bfff640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe86006be30 0x7fe86006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:51.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.954+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe8680ab7a0 con 0x7fe8740ab310 2024-05-09T07:17:51.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.954+0000 7fe8737fe640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe86006be30 0x7fe86006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:51.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.955+0000 7fe8737fe640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe86006be30 0x7fe86006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8740a96b0 tx=0x7fe864009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:51.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:51.959+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe86804b030 con 0x7fe8740ab310 2024-05-09T07:17:51.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 384083 15.89 MB/sec execute 24 sec latency 20.440 ms 2024-05-09T07:17:52.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.161+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fe8740c1b50 con 0x7fe8740ab310 2024-05-09T07:17:52.165 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.161+0000 7fe85bfff640 1 -- 172.21.15.160:0/477618733 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7fe8680797a0 con 0x7fe8740ab310 2024-05-09T07:17:52.165 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:17:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe86006be30 msgr2=0x7fe86006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe86006be30 0x7fe86006e2f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe8740a96b0 tx=0x7fe864009190 comp rx=0 tx=0).stop 2024-05-09T07:17:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 msgr2=0x7fe874144920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 0x7fe874144920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe868035040 tx=0x7fe8680046c0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 shutdown_connections 2024-05-09T07:17:52.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe86006be30 0x7fe86006e2f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe8741550e0 0x7fe87414c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8740ab310 0x7fe874144920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 --2- 172.21.15.160:0/477618733 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe87406d510 0x7fe8741e9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 >> 172.21.15.160:0/477618733 conn(0x7fe8740c69b0 msgr2=0x7fe8740c5120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:52.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.163+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 shutdown_connections 2024-05-09T07:17:52.168 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.164+0000 7fe879d4f640 1 -- 172.21.15.160:0/477618733 wait complete. 2024-05-09T07:17:52.178 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:17:52.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.282+0000 7ffbe6238640 1 Processor -- start 2024-05-09T07:17:52.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.282+0000 7ffbe6238640 1 -- start start 2024-05-09T07:17:52.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbe6238640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7ffbe00a6270 0x7ffbe00c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbe6238640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffbe00aa1d0 0x7ffbe00aa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbe6238640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffbe0155710 con 0x7ffbe00c59c0 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbe6238640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe00aaba0 con 0x7ffbe00aa1d0 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbe6238640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7ffbe006dab0 con 0x7ffbe00a6270 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffbe00aa1d0 0x7ffbe00aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdf7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7ffbe00c59c0 0x7ffbe006d240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:39506/0 (socket says 172.21.15.160:39506) 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdffff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7ffbe00aa1d0 0x7ffbe00aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34650/0 (socket says 172.21.15.160:34650) 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdf7fe640 1 -- 172.21.15.160:0/845002422 learned_addr learned my addr 172.21.15.160:0/845002422 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdeffd640 1 --2- 172.21.15.160:0/845002422 >> v2:172.21.15.188:3300/0 conn(0x7ffbe00a6270 0x7ffbe00c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdffff640 1 -- 172.21.15.160:0/845002422 >> v2:172.21.15.188:3300/0 conn(0x7ffbe00a6270 msgr2=0x7ffbe00c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdffff640 1 --2- 172.21.15.160:0/845002422 >> v2:172.21.15.188:3300/0 conn(0x7ffbe00a6270 0x7ffbe00c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdffff640 1 -- 172.21.15.160:0/845002422 >> v1:172.21.15.160:6789/0 conn(0x7ffbe00c59c0 legacy=0x7ffbe006d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbdffff640 1 -- 172.21.15.160:0/845002422 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe01559b0 con 0x7ffbe00aa1d0 2024-05-09T07:17:52.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.283+0000 7ffbde7fc640 1 -- 172.21.15.160:0/845002422 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 787700439 0 0) 0x7ffbe0155710 con 0x7ffbe00c59c0 2024-05-09T07:17:52.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.284+0000 7ffbdffff640 1 --2- 172.21.15.160:0/845002422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00aa1d0 0x7ffbe00aa660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0070850 tx=0x7ffbd802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f288e452286c025 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:52.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.284+0000 7ffbde7fc640 1 -- 172.21.15.160:0/845002422 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8007740 con 0x7ffbe00aa1d0 2024-05-09T07:17:52.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.284+0000 7ffbde7fc640 1 -- 172.21.15.160:0/845002422 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd80078e0 con 0x7ffbe00aa1d0 2024-05-09T07:17:52.289 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.284+0000 7ffbde7fc640 1 -- 172.21.15.160:0/845002422 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd8007cb0 con 0x7ffbe00aa1d0 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 -- 172.21.15.160:0/845002422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00aa1d0 msgr2=0x7ffbe00aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 --2- 172.21.15.160:0/845002422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00aa1d0 0x7ffbe00aa660 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0070850 tx=0x7ffbd802a680 comp rx=0 tx=0).stop 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 -- 172.21.15.160:0/845002422 shutdown_connections 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 --2- 172.21.15.160:0/845002422 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00aa1d0 0x7ffbe00aa660 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 --2- 172.21.15.160:0/845002422 >> v2:172.21.15.188:3300/0 conn(0x7ffbe00a6270 0x7ffbe00c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 -- 172.21.15.160:0/845002422 >> 172.21.15.160:0/845002422 conn(0x7ffbe00c69b0 msgr2=0x7ffbe00c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 -- 172.21.15.160:0/845002422 shutdown_connections 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.285+0000 7ffbe6238640 1 -- 172.21.15.160:0/845002422 wait complete. 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.286+0000 7ffbe6238640 1 Processor -- start 2024-05-09T07:17:52.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.286+0000 7ffbe6238640 1 -- start start 2024-05-09T07:17:52.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbe6238640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 0x7ffbe01dfe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbe6238640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbe00aa1d0 0x7ffbe0072440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbe6238640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbe00c59c0 0x7ffbe00708a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbe6238640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe014fa20 con 0x7ffbe00a6270 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbe6238640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe0155f70 con 0x7ffbe00c59c0 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbe6238640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe014fba0 con 0x7ffbe00aa1d0 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 0x7ffbe01dfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 0x7ffbe01dfe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34654/0 (socket says 172.21.15.160:34654) 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 -- 172.21.15.160:0/2957579536 learned_addr learned my addr 172.21.15.160:0/2957579536 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdeffd640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbe00aa1d0 0x7ffbe0072440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdffff640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbe00c59c0 0x7ffbe00708a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 -- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbe00aa1d0 msgr2=0x7ffbe0072440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbe00aa1d0 0x7ffbe0072440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 -- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbe00c59c0 msgr2=0x7ffbe00708a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbe00c59c0 0x7ffbe00708a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 -- 172.21.15.160:0/2957579536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe00c22f0 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbdf7fe640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 0x7ffbe01dfe00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c430 tx=0x7ffbd000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.287+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd00074e0 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.288+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffbd00047e0 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.288+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe00c2590 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.288+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0004bd0 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.288+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe014fed0 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.289+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbe00c2760 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.289+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7ffbd0011070 con 0x7ffbe00a6270 2024-05-09T07:17:52.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.289+0000 7ffbc7fff640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ffbcc06be30 0x7ffbcc06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.290+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7ffbd008ac90 con 0x7ffbe00a6270 2024-05-09T07:17:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.290+0000 7ffbdeffd640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ffbcc06be30 0x7ffbcc06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.295 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.290+0000 7ffbdeffd640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ffbcc06be30 0x7ffbcc06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffbe014dc30 tx=0x7ffbd4009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:52.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.294+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ffbd0058c90 con 0x7ffbe00a6270 2024-05-09T07:17:52.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.471+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffbe0075650 con 0x7ffbe00a6270 2024-05-09T07:17:52.475 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.472+0000 7ffbc7fff640 1 -- 172.21.15.160:0/2957579536 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ffbd005cf90 con 0x7ffbe00a6270 2024-05-09T07:17:52.476 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:17:52.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ffbcc06be30 msgr2=0x7ffbcc06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ffbcc06be30 0x7ffbcc06e2f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ffbe014dc30 tx=0x7ffbd4009190 comp rx=0 tx=0).stop 2024-05-09T07:17:52.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 msgr2=0x7ffbe01dfe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 0x7ffbe01dfe00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c430 tx=0x7ffbd000c8f0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 shutdown_connections 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7ffbcc06be30 0x7ffbcc06e2f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7ffbe00c59c0 0x7ffbe00708a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7ffbe00aa1d0 0x7ffbe0072440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 --2- 172.21.15.160:0/2957579536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffbe00a6270 0x7ffbe01dfe00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 >> 172.21.15.160:0/2957579536 conn(0x7ffbe00c69b0 msgr2=0x7ffbe00c48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 shutdown_connections 2024-05-09T07:17:52.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.474+0000 7ffbe6238640 1 -- 172.21.15.160:0/2957579536 wait complete. 2024-05-09T07:17:52.490 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T07:17:52.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.588+0000 7f8208b77640 1 Processor -- start 2024-05-09T07:17:52.592 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.589+0000 7f8208b77640 1 -- start start 2024-05-09T07:17:52.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.589+0000 7f8208b77640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f82040c6460 0x7f82040c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.589+0000 7f8208b77640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82040c47d0 con 0x7f82040c19f0 2024-05-09T07:17:52.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.589+0000 7f8208b77640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8204152920 con 0x7f820406c670 2024-05-09T07:17:52.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.589+0000 7f8208b77640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f82040bdb40 con 0x7f82040c6460 2024-05-09T07:17:52.594 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8201d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f82040c6460 0x7f82040c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8202d9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f82040c19f0 0x7f82040c2c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:55258/0 (socket says 172.21.15.160:55258) 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8201d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f82040c6460 0x7f82040c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51562/0 (socket says 172.21.15.160:51562) 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8202d9d640 1 -- 172.21.15.160:0/345053773 learned_addr learned my addr 172.21.15.160:0/345053773 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8201d9b640 1 -- 172.21.15.160:0/345053773 >> v1:172.21.15.188:6789/0 conn(0x7f820406c670 legacy=0x7f820406ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8201d9b640 1 -- 172.21.15.160:0/345053773 >> v1:172.21.15.160:6789/0 conn(0x7f82040c19f0 legacy=0x7f82040c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f820159a640 1 -- 172.21.15.160:0/345053773 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3358446079 0 0) 0x7f8204152920 con 0x7f820406c670 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.590+0000 7f8201d9b640 1 -- 172.21.15.160:0/345053773 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82040a6170 con 0x7f82040c6460 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8201d9b640 1 --2- 172.21.15.160:0/345053773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c6460 0x7f82040c2720 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0097a0 tx=0x7f81ec02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b01ad0faa0c9a93 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f820159a640 1 -- 172.21.15.160:0/345053773 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec037070 con 0x7f82040c6460 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f820159a640 1 -- 172.21.15.160:0/345053773 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec02ac90 con 0x7f82040c6460 2024-05-09T07:17:52.595 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f820159a640 1 -- 172.21.15.160:0/345053773 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec032660 con 0x7f82040c6460 2024-05-09T07:17:52.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 -- 172.21.15.160:0/345053773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c6460 msgr2=0x7f82040c2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 --2- 172.21.15.160:0/345053773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c6460 0x7f82040c2720 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f81ec0097a0 tx=0x7f81ec02a000 comp rx=0 tx=0).stop 2024-05-09T07:17:52.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 -- 172.21.15.160:0/345053773 shutdown_connections 2024-05-09T07:17:52.596 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 --2- 172.21.15.160:0/345053773 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c6460 0x7f82040c2720 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 -- 172.21.15.160:0/345053773 >> 172.21.15.160:0/345053773 conn(0x7f82040c7580 msgr2=0x7f82040a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:52.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 -- 172.21.15.160:0/345053773 shutdown_connections 2024-05-09T07:17:52.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.591+0000 7f8208b77640 1 -- 172.21.15.160:0/345053773 wait complete. 2024-05-09T07:17:52.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 Processor -- start 2024-05-09T07:17:52.597 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 -- start start 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f820406c670 0x7f8204149660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 0x7f82040bf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040c6460 0x7f82040be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f82040c2200 con 0x7f82040c19f0 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f82041523f0 con 0x7f820406c670 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8208b77640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f82040c14f0 con 0x7f82040c6460 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8201d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 0x7f82040bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8201d9b640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 0x7f82040bf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:51574/0 (socket says 172.21.15.160:51574) 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f8201d9b640 1 -- 172.21.15.160:0/4081816392 learned_addr learned my addr 172.21.15.160:0/4081816392 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.593+0000 7f820259c640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f820406c670 0x7f8204149660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8202d9d640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040c6460 0x7f82040be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.599 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8201d9b640 1 -- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040c6460 msgr2=0x7f82040be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8201d9b640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040c6460 0x7f82040be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8201d9b640 1 -- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f820406c670 msgr2=0x7f8204149660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8201d9b640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f820406c670 0x7f8204149660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8201d9b640 1 -- 172.21.15.160:0/4081816392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82040bdcd0 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8201d9b640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 0x7f82040bf180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000c00 tx=0x7f81ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec037040 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81ec004030 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81ec004420 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82040727e0 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.594+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8204072da0 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.595+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82040756c0 con 0x7f82040c19f0 2024-05-09T07:17:52.600 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.596+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f81ec03a070 con 0x7f82040c19f0 2024-05-09T07:17:52.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.596+0000 7f81faffd640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f81e406be60 0x7f81e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:17:52.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.596+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f81ec0ab700 con 0x7f82040c19f0 2024-05-09T07:17:52.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.596+0000 7f820259c640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f81e406be60 0x7f81e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:17:52.601 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.596+0000 7f820259c640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f81e406be60 0x7f81e406e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000f60 tx=0x7f81f4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:17:52.604 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.600+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f81ec041020 con 0x7f82040c19f0 2024-05-09T07:17:52.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.788+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f820406beb0 con 0x7f82040c19f0 2024-05-09T07:17:52.792 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.788+0000 7f81faffd640 1 -- 172.21.15.160:0/4081816392 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f81ec079670 con 0x7f82040c19f0 2024-05-09T07:17:52.792 INFO:teuthology.orchestra.run.smithi160.stderr:listed 0 entries 2024-05-09T07:17:52.794 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f81e406be60 msgr2=0x7f81e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f81e406be60 0x7f81e406e320 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f81f4000f60 tx=0x7f81f4009040 comp rx=0 tx=0).stop 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 msgr2=0x7f82040bf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 0x7f82040bf180 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81ec000c00 tx=0x7f81ec0046c0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 shutdown_connections 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f81e406be60 0x7f81e406e320 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f82040c6460 0x7f82040be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f82040c19f0 0x7f82040bf180 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 --2- 172.21.15.160:0/4081816392 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f820406c670 0x7f8204149660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 >> 172.21.15.160:0/4081816392 conn(0x7f82040c7580 msgr2=0x7f820406ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 shutdown_connections 2024-05-09T07:17:52.795 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:17:52.791+0000 7f8208b77640 1 -- 172.21.15.160:0/4081816392 wait complete. 2024-05-09T07:17:52.805 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi160.front.sepia.ceph.com... 2024-05-09T07:17:52.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:17:52.806 DEBUG:teuthology.orchestra.run.smithi160:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-05-09T07:17:52.832 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:17:52.833 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi160.stderr:ceph-fuse[46508]: fuse finished with error 0 and tester_r 0 2024-05-09T07:17:52.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 387416 16.00 MB/sec execute 25 sec latency 49.189 ms 2024-05-09T07:17:53.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 390796 16.10 MB/sec execute 26 sec latency 24.791 ms 2024-05-09T07:17:54.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 394303 16.22 MB/sec execute 27 sec latency 13.071 ms 2024-05-09T07:17:55.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 397771 16.32 MB/sec execute 28 sec latency 16.112 ms 2024-05-09T07:17:56.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 401169 16.40 MB/sec execute 29 sec latency 12.957 ms 2024-05-09T07:17:57.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 404519 16.46 MB/sec execute 30 sec latency 33.445 ms 2024-05-09T07:17:58.934 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:17:58.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:17:58.934 DEBUG:teuthology.orchestra.run.smithi160:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T07:17:58.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:17:58.950 DEBUG:teuthology.orchestra.run.smithi160:> ip netns list 2024-05-09T07:17:58.963 INFO:tasks.workunit.client.1.smithi188.stdout: 1 407936 16.54 MB/sec execute 31 sec latency 28.945 ms 2024-05-09T07:17:59.006 INFO:teuthology.orchestra.run.smithi160.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T07:17:59.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:17:59.007 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T07:17:59.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:17:59.083 DEBUG:teuthology.orchestra.run.smithi160:> sudo ip link delete ceph-brx 2024-05-09T07:17:59.183 DEBUG:teuthology.parallel:result is None 2024-05-09T07:17:59.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 411456 16.62 MB/sec execute 32 sec latency 18.451 ms 2024-05-09T07:18:00.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 414761 16.60 MB/sec execute 33 sec latency 26.945 ms 2024-05-09T07:18:01.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 417818 16.57 MB/sec execute 34 sec latency 22.515 ms 2024-05-09T07:18:02.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 421051 16.55 MB/sec execute 35 sec latency 16.718 ms 2024-05-09T07:18:03.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 424025 16.56 MB/sec execute 36 sec latency 28.166 ms 2024-05-09T07:18:04.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 427311 16.62 MB/sec execute 37 sec latency 19.956 ms 2024-05-09T07:18:05.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 430674 16.69 MB/sec execute 38 sec latency 27.535 ms 2024-05-09T07:18:06.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 434049 16.67 MB/sec execute 39 sec latency 16.640 ms 2024-05-09T07:18:07.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 435101 16.37 MB/sec execute 40 sec latency 180.690 ms 2024-05-09T07:18:08.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 436701 16.27 MB/sec execute 41 sec latency 113.181 ms 2024-05-09T07:18:09.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 440247 16.34 MB/sec execute 42 sec latency 10.488 ms 2024-05-09T07:18:10.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 443728 16.40 MB/sec execute 43 sec latency 17.529 ms 2024-05-09T07:18:11.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 447198 16.46 MB/sec execute 44 sec latency 32.371 ms 2024-05-09T07:18:12.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 450631 16.50 MB/sec execute 45 sec latency 32.613 ms 2024-05-09T07:18:13.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 454155 16.55 MB/sec execute 46 sec latency 24.687 ms 2024-05-09T07:18:14.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 457700 16.61 MB/sec execute 47 sec latency 18.624 ms 2024-05-09T07:18:15.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 460671 16.88 MB/sec execute 48 sec latency 40.342 ms 2024-05-09T07:18:16.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 464351 16.93 MB/sec execute 49 sec latency 19.377 ms 2024-05-09T07:18:17.097 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:18:17.102 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:18:17.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 467937 16.98 MB/sec execute 50 sec latency 19.330 ms 2024-05-09T07:18:18.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 471404 17.01 MB/sec execute 51 sec latency 54.602 ms 2024-05-09T07:18:19.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 474956 17.05 MB/sec execute 52 sec latency 11.624 ms 2024-05-09T07:18:20.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 478230 17.07 MB/sec execute 53 sec latency 56.724 ms 2024-05-09T07:18:21.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 481444 17.10 MB/sec execute 54 sec latency 13.019 ms 2024-05-09T07:18:22.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 483958 17.04 MB/sec execute 55 sec latency 229.051 ms 2024-05-09T07:18:23.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 487444 17.08 MB/sec execute 56 sec latency 31.957 ms 2024-05-09T07:18:24.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 491053 17.12 MB/sec execute 57 sec latency 11.817 ms 2024-05-09T07:18:25.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 494607 17.15 MB/sec execute 58 sec latency 13.200 ms 2024-05-09T07:18:26.964 INFO:tasks.workunit.client.1.smithi188.stdout: 1 498006 17.18 MB/sec execute 59 sec latency 36.276 ms 2024-05-09T07:18:27.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 500782 17.13 MB/sec execute 60 sec latency 230.446 ms 2024-05-09T07:18:28.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 501922 16.94 MB/sec execute 61 sec latency 295.237 ms 2024-05-09T07:18:29.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 504596 16.94 MB/sec execute 62 sec latency 43.002 ms 2024-05-09T07:18:30.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 508038 16.97 MB/sec execute 63 sec latency 18.294 ms 2024-05-09T07:18:31.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 511240 16.94 MB/sec execute 64 sec latency 17.575 ms 2024-05-09T07:18:32.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 514366 16.94 MB/sec execute 65 sec latency 27.397 ms 2024-05-09T07:18:33.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 517591 16.92 MB/sec execute 66 sec latency 23.685 ms 2024-05-09T07:18:34.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 521103 16.95 MB/sec execute 67 sec latency 18.641 ms 2024-05-09T07:18:35.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 524131 16.95 MB/sec execute 68 sec latency 12.718 ms 2024-05-09T07:18:36.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 527423 16.98 MB/sec execute 69 sec latency 17.502 ms 2024-05-09T07:18:37.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 530776 17.01 MB/sec execute 70 sec latency 22.428 ms 2024-05-09T07:18:38.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 533814 16.98 MB/sec execute 71 sec latency 19.860 ms 2024-05-09T07:18:39.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 537231 17.01 MB/sec execute 72 sec latency 35.957 ms 2024-05-09T07:18:40.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 540731 17.03 MB/sec execute 73 sec latency 15.714 ms 2024-05-09T07:18:41.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 544132 17.06 MB/sec execute 74 sec latency 24.303 ms 2024-05-09T07:18:42.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 547429 17.08 MB/sec execute 75 sec latency 47.122 ms 2024-05-09T07:18:43.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 550906 17.11 MB/sec execute 76 sec latency 19.341 ms 2024-05-09T07:18:44.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 554370 17.13 MB/sec execute 77 sec latency 16.401 ms 2024-05-09T07:18:45.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 557768 17.14 MB/sec execute 78 sec latency 29.401 ms 2024-05-09T07:18:46.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 561193 17.17 MB/sec execute 79 sec latency 34.988 ms 2024-05-09T07:18:47.388 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:18:47.391 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:18:47.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 564185 17.13 MB/sec execute 80 sec latency 49.565 ms 2024-05-09T07:18:48.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 565100 17.00 MB/sec execute 81 sec latency 216.385 ms 2024-05-09T07:18:49.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 566282 16.85 MB/sec execute 82 sec latency 41.611 ms 2024-05-09T07:18:50.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 569832 16.88 MB/sec execute 83 sec latency 15.305 ms 2024-05-09T07:18:51.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 572991 16.87 MB/sec execute 84 sec latency 26.060 ms 2024-05-09T07:18:52.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 576246 16.88 MB/sec execute 85 sec latency 36.515 ms 2024-05-09T07:18:53.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 579666 16.90 MB/sec execute 86 sec latency 20.177 ms 2024-05-09T07:18:54.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 583146 16.93 MB/sec execute 87 sec latency 30.829 ms 2024-05-09T07:18:55.965 INFO:tasks.workunit.client.1.smithi188.stdout: 1 586511 16.95 MB/sec execute 88 sec latency 14.444 ms 2024-05-09T07:18:56.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 589925 16.97 MB/sec execute 89 sec latency 22.152 ms 2024-05-09T07:18:57.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 593235 16.98 MB/sec execute 90 sec latency 29.094 ms 2024-05-09T07:18:58.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 596691 17.00 MB/sec execute 91 sec latency 27.115 ms 2024-05-09T07:18:59.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 599922 16.98 MB/sec execute 92 sec latency 20.019 ms 2024-05-09T07:19:00.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 603077 16.98 MB/sec execute 93 sec latency 20.897 ms 2024-05-09T07:19:01.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 606200 16.96 MB/sec execute 94 sec latency 11.329 ms 2024-05-09T07:19:02.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 609039 16.96 MB/sec execute 95 sec latency 24.178 ms 2024-05-09T07:19:03.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 612329 16.98 MB/sec execute 96 sec latency 15.511 ms 2024-05-09T07:19:04.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 615847 17.01 MB/sec execute 97 sec latency 15.260 ms 2024-05-09T07:19:05.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 619393 17.03 MB/sec execute 98 sec latency 19.662 ms 2024-05-09T07:19:06.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 622586 17.02 MB/sec execute 99 sec latency 20.398 ms 2024-05-09T07:19:07.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 626056 17.03 MB/sec execute 100 sec latency 24.571 ms 2024-05-09T07:19:08.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 629560 17.05 MB/sec execute 101 sec latency 23.201 ms 2024-05-09T07:19:09.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 632967 17.07 MB/sec execute 102 sec latency 30.561 ms 2024-05-09T07:19:10.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 636329 17.08 MB/sec execute 103 sec latency 19.839 ms 2024-05-09T07:19:11.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 639673 17.10 MB/sec execute 104 sec latency 21.457 ms 2024-05-09T07:19:12.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 643003 17.12 MB/sec execute 105 sec latency 18.447 ms 2024-05-09T07:19:13.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 646336 17.12 MB/sec execute 106 sec latency 39.989 ms 2024-05-09T07:19:14.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 649544 17.10 MB/sec execute 107 sec latency 22.504 ms 2024-05-09T07:19:15.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 652889 17.11 MB/sec execute 108 sec latency 30.238 ms 2024-05-09T07:19:16.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 656212 17.12 MB/sec execute 109 sec latency 37.318 ms 2024-05-09T07:19:17.724 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:19:17.727 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:19:17.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 658669 17.08 MB/sec execute 110 sec latency 46.495 ms 2024-05-09T07:19:18.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 661787 17.07 MB/sec execute 111 sec latency 11.225 ms 2024-05-09T07:19:19.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 664806 17.08 MB/sec execute 112 sec latency 130.789 ms 2024-05-09T07:19:20.966 INFO:tasks.workunit.client.1.smithi188.stdout: 1 664828 16.93 MB/sec execute 113 sec latency 494.639 ms 2024-05-09T07:19:21.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 666674 16.84 MB/sec execute 114 sec latency 158.192 ms 2024-05-09T07:19:22.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 668364 16.80 MB/sec execute 115 sec latency 50.416 ms 2024-05-09T07:19:23.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 671829 16.82 MB/sec execute 116 sec latency 65.100 ms 2024-05-09T07:19:24.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 675235 16.83 MB/sec execute 117 sec latency 27.810 ms 2024-05-09T07:19:25.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 678678 16.85 MB/sec execute 118 sec latency 31.697 ms 2024-05-09T07:19:26.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 682095 16.87 MB/sec execute 119 sec latency 32.947 ms 2024-05-09T07:19:27.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 685193 16.85 MB/sec execute 120 sec latency 44.356 ms 2024-05-09T07:19:28.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 688116 16.84 MB/sec execute 121 sec latency 22.893 ms 2024-05-09T07:19:29.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 691333 16.83 MB/sec execute 122 sec latency 31.159 ms 2024-05-09T07:19:30.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 693979 16.83 MB/sec execute 123 sec latency 17.009 ms 2024-05-09T07:19:31.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 697030 16.82 MB/sec execute 124 sec latency 20.217 ms 2024-05-09T07:19:32.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 700271 16.83 MB/sec execute 125 sec latency 28.839 ms 2024-05-09T07:19:33.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 703599 16.85 MB/sec execute 126 sec latency 27.516 ms 2024-05-09T07:19:34.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 706991 16.86 MB/sec execute 127 sec latency 15.201 ms 2024-05-09T07:19:35.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 710430 16.88 MB/sec execute 128 sec latency 24.667 ms 2024-05-09T07:19:36.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 713598 16.86 MB/sec execute 129 sec latency 23.812 ms 2024-05-09T07:19:37.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 716732 16.87 MB/sec execute 130 sec latency 60.917 ms 2024-05-09T07:19:38.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 719761 16.84 MB/sec execute 131 sec latency 83.470 ms 2024-05-09T07:19:39.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 722515 16.84 MB/sec execute 132 sec latency 25.420 ms 2024-05-09T07:19:40.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 724383 16.81 MB/sec execute 133 sec latency 491.161 ms 2024-05-09T07:19:41.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 725348 16.70 MB/sec execute 134 sec latency 104.628 ms 2024-05-09T07:19:42.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 728418 16.71 MB/sec execute 135 sec latency 33.105 ms 2024-05-09T07:19:43.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 731786 16.73 MB/sec execute 136 sec latency 25.924 ms 2024-05-09T07:19:44.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 735020 16.74 MB/sec execute 137 sec latency 26.322 ms 2024-05-09T07:19:45.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 738224 16.72 MB/sec execute 138 sec latency 26.854 ms 2024-05-09T07:19:46.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 741345 16.73 MB/sec execute 139 sec latency 18.422 ms 2024-05-09T07:19:47.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 744688 16.72 MB/sec execute 140 sec latency 22.590 ms 2024-05-09T07:19:48.075 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:19:48.077 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:19:48.123 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-05-09T07:19:48.119+0000 7f44cf7fa640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42687) UID: 0 2024-05-09T07:19:48.124 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-05-09T07:19:48.119+0000 7fa6c2e0d640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42687) UID: 0 2024-05-09T07:19:48.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 747484 16.72 MB/sec execute 141 sec latency 36.893 ms 2024-05-09T07:19:49.967 INFO:tasks.workunit.client.1.smithi188.stdout: 1 750163 16.70 MB/sec execute 142 sec latency 35.132 ms 2024-05-09T07:19:50.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 753642 16.72 MB/sec execute 143 sec latency 25.807 ms 2024-05-09T07:19:51.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 756984 16.73 MB/sec execute 144 sec latency 26.049 ms 2024-05-09T07:19:52.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 760293 16.75 MB/sec execute 145 sec latency 24.675 ms 2024-05-09T07:19:53.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 763771 16.77 MB/sec execute 146 sec latency 19.517 ms 2024-05-09T07:19:54.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 767257 16.78 MB/sec execute 147 sec latency 28.564 ms 2024-05-09T07:19:55.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 770722 16.79 MB/sec execute 148 sec latency 17.366 ms 2024-05-09T07:19:56.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 774012 16.80 MB/sec execute 149 sec latency 28.230 ms 2024-05-09T07:19:57.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 776814 16.77 MB/sec execute 150 sec latency 34.172 ms 2024-05-09T07:19:58.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 780041 16.76 MB/sec execute 151 sec latency 21.641 ms 2024-05-09T07:19:59.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 782788 16.77 MB/sec execute 152 sec latency 11.909 ms 2024-05-09T07:20:00.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 785360 16.75 MB/sec execute 153 sec latency 114.906 ms 2024-05-09T07:20:01.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 785941 16.65 MB/sec execute 154 sec latency 411.125 ms 2024-05-09T07:20:02.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 788506 16.65 MB/sec execute 155 sec latency 56.066 ms 2024-05-09T07:20:03.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 790665 16.59 MB/sec execute 156 sec latency 26.297 ms 2024-05-09T07:20:04.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 793207 16.60 MB/sec execute 157 sec latency 12.941 ms 2024-05-09T07:20:05.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 796038 16.58 MB/sec execute 158 sec latency 19.222 ms 2024-05-09T07:20:06.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 799154 16.59 MB/sec execute 159 sec latency 13.813 ms 2024-05-09T07:20:07.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 802158 16.58 MB/sec execute 160 sec latency 23.375 ms 2024-05-09T07:20:08.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 805304 16.58 MB/sec execute 161 sec latency 23.506 ms 2024-05-09T07:20:09.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 807649 16.56 MB/sec execute 162 sec latency 24.451 ms 2024-05-09T07:20:10.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 810569 16.55 MB/sec execute 163 sec latency 20.394 ms 2024-05-09T07:20:11.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 813847 16.56 MB/sec execute 164 sec latency 28.790 ms 2024-05-09T07:20:12.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 817213 16.57 MB/sec execute 165 sec latency 27.426 ms 2024-05-09T07:20:13.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 820415 16.59 MB/sec execute 166 sec latency 31.269 ms 2024-05-09T07:20:14.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 823633 16.59 MB/sec execute 167 sec latency 29.075 ms 2024-05-09T07:20:15.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 826579 16.58 MB/sec execute 168 sec latency 22.550 ms 2024-05-09T07:20:16.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 829606 16.56 MB/sec execute 169 sec latency 15.611 ms 2024-05-09T07:20:17.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 832434 16.57 MB/sec execute 170 sec latency 38.592 ms 2024-05-09T07:20:18.968 INFO:tasks.workunit.client.1.smithi188.stdout: 1 835728 16.58 MB/sec execute 171 sec latency 26.736 ms 2024-05-09T07:20:19.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 838712 16.57 MB/sec execute 172 sec latency 24.167 ms 2024-05-09T07:20:20.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 842115 16.58 MB/sec execute 173 sec latency 21.001 ms 2024-05-09T07:20:21.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 845582 16.60 MB/sec execute 174 sec latency 29.711 ms 2024-05-09T07:20:22.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 848950 16.61 MB/sec execute 175 sec latency 27.914 ms 2024-05-09T07:20:23.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 852247 16.62 MB/sec execute 176 sec latency 21.326 ms 2024-05-09T07:20:24.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 855091 16.60 MB/sec execute 177 sec latency 21.796 ms 2024-05-09T07:20:25.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 858151 16.59 MB/sec execute 178 sec latency 22.159 ms 2024-05-09T07:20:26.969 INFO:tasks.workunit.client.1.smithi188.stdout: 1 859586 16.56 MB/sec execute 179 sec latency 551.492 ms 2024-05-09T07:20:27.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 860146 16.47 MB/sec execute 180 sec latency 781.495 ms 2024-05-09T07:20:28.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 862404 16.45 MB/sec execute 181 sec latency 29.134 ms 2024-05-09T07:20:29.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 865146 16.43 MB/sec execute 182 sec latency 28.604 ms 2024-05-09T07:20:30.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 867544 16.42 MB/sec execute 183 sec latency 24.649 ms 2024-05-09T07:20:31.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 870575 16.42 MB/sec execute 184 sec latency 36.318 ms 2024-05-09T07:20:32.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 873764 16.44 MB/sec execute 185 sec latency 47.471 ms 2024-05-09T07:20:33.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 876760 16.42 MB/sec execute 186 sec latency 35.052 ms 2024-05-09T07:20:34.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 879826 16.43 MB/sec execute 187 sec latency 23.813 ms 2024-05-09T07:20:35.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 882776 16.41 MB/sec execute 188 sec latency 23.484 ms 2024-05-09T07:20:36.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 885125 16.40 MB/sec execute 189 sec latency 18.213 ms 2024-05-09T07:20:37.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 888289 16.41 MB/sec execute 190 sec latency 21.724 ms 2024-05-09T07:20:38.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 891511 16.42 MB/sec execute 191 sec latency 21.124 ms 2024-05-09T07:20:39.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 894646 16.42 MB/sec execute 192 sec latency 28.973 ms 2024-05-09T07:20:40.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 897148 16.39 MB/sec execute 193 sec latency 66.920 ms 2024-05-09T07:20:41.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 899758 16.39 MB/sec execute 194 sec latency 68.312 ms 2024-05-09T07:20:42.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 902728 16.38 MB/sec execute 195 sec latency 17.503 ms 2024-05-09T07:20:43.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 905910 16.39 MB/sec execute 196 sec latency 20.519 ms 2024-05-09T07:20:44.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 909162 16.40 MB/sec execute 197 sec latency 19.990 ms 2024-05-09T07:20:45.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 912470 16.41 MB/sec execute 198 sec latency 38.363 ms 2024-05-09T07:20:46.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 915363 16.40 MB/sec execute 199 sec latency 23.179 ms 2024-05-09T07:20:47.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 917352 16.46 MB/sec execute 200 sec latency 67.314 ms 2024-05-09T07:20:48.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 917416 16.38 MB/sec execute 201 sec latency 463.598 ms 2024-05-09T07:20:49.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 919688 16.36 MB/sec execute 202 sec latency 35.005 ms 2024-05-09T07:20:50.970 INFO:tasks.workunit.client.1.smithi188.stdout: 1 923123 16.36 MB/sec execute 203 sec latency 32.526 ms 2024-05-09T07:20:51.975 INFO:tasks.workunit.client.1.smithi188.stdout: 1 926226 16.36 MB/sec execute 204 sec latency 24.025 ms 2024-05-09T07:20:52.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 929695 16.37 MB/sec execute 205 sec latency 24.751 ms 2024-05-09T07:20:53.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 932135 16.36 MB/sec execute 206 sec latency 35.393 ms 2024-05-09T07:20:54.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 935223 16.36 MB/sec execute 207 sec latency 21.579 ms 2024-05-09T07:20:55.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 938560 16.37 MB/sec execute 208 sec latency 26.101 ms 2024-05-09T07:20:56.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 941863 16.39 MB/sec execute 209 sec latency 14.034 ms 2024-05-09T07:20:57.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 945250 16.39 MB/sec execute 210 sec latency 31.184 ms 2024-05-09T07:20:58.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 948482 16.39 MB/sec execute 211 sec latency 35.258 ms 2024-05-09T07:20:59.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 951529 16.39 MB/sec execute 212 sec latency 19.480 ms 2024-05-09T07:21:00.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 954613 16.38 MB/sec execute 213 sec latency 32.917 ms 2024-05-09T07:21:01.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 957276 16.39 MB/sec execute 214 sec latency 16.595 ms 2024-05-09T07:21:02.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 960222 16.38 MB/sec execute 215 sec latency 32.179 ms 2024-05-09T07:21:03.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 963409 16.39 MB/sec execute 216 sec latency 31.595 ms 2024-05-09T07:21:04.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 966646 16.40 MB/sec execute 217 sec latency 21.087 ms 2024-05-09T07:21:05.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 969875 16.40 MB/sec execute 218 sec latency 10.906 ms 2024-05-09T07:21:06.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 972360 16.37 MB/sec execute 219 sec latency 32.114 ms 2024-05-09T07:21:07.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 974412 16.35 MB/sec execute 220 sec latency 30.590 ms 2024-05-09T07:21:08.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 977451 16.36 MB/sec execute 221 sec latency 32.717 ms 2024-05-09T07:21:09.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 980667 16.37 MB/sec execute 222 sec latency 24.467 ms 2024-05-09T07:21:10.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 983178 16.34 MB/sec execute 223 sec latency 14.781 ms 2024-05-09T07:21:11.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 986525 16.35 MB/sec execute 224 sec latency 35.185 ms 2024-05-09T07:21:12.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 989346 16.35 MB/sec execute 225 sec latency 28.682 ms 2024-05-09T07:21:13.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 992304 16.35 MB/sec execute 226 sec latency 43.044 ms 2024-05-09T07:21:14.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 995519 16.35 MB/sec execute 227 sec latency 36.118 ms 2024-05-09T07:21:15.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 998828 16.36 MB/sec execute 228 sec latency 25.923 ms 2024-05-09T07:21:16.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1002007 16.37 MB/sec execute 229 sec latency 20.363 ms 2024-05-09T07:21:17.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1004888 16.36 MB/sec execute 230 sec latency 21.830 ms 2024-05-09T07:21:18.971 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1008023 16.35 MB/sec execute 231 sec latency 28.382 ms 2024-05-09T07:21:19.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1011294 16.36 MB/sec execute 232 sec latency 25.388 ms 2024-05-09T07:21:20.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1013914 16.36 MB/sec execute 233 sec latency 29.248 ms 2024-05-09T07:21:21.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1016933 16.36 MB/sec execute 234 sec latency 23.810 ms 2024-05-09T07:21:22.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1020276 16.37 MB/sec execute 235 sec latency 29.885 ms 2024-05-09T07:21:23.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1023262 16.37 MB/sec execute 236 sec latency 111.353 ms 2024-05-09T07:21:24.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1023280 16.30 MB/sec execute 237 sec latency 616.603 ms 2024-05-09T07:21:25.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1023928 16.24 MB/sec execute 238 sec latency 408.932 ms 2024-05-09T07:21:26.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1024668 16.19 MB/sec execute 239 sec latency 77.015 ms 2024-05-09T07:21:27.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1027479 16.19 MB/sec execute 240 sec latency 27.239 ms 2024-05-09T07:21:28.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1030611 16.20 MB/sec execute 241 sec latency 38.925 ms 2024-05-09T07:21:29.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1033848 16.20 MB/sec execute 242 sec latency 17.307 ms 2024-05-09T07:21:30.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1036374 16.17 MB/sec execute 243 sec latency 38.308 ms 2024-05-09T07:21:31.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1039025 16.17 MB/sec execute 244 sec latency 35.542 ms 2024-05-09T07:21:32.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1041850 16.17 MB/sec execute 245 sec latency 27.600 ms 2024-05-09T07:21:33.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1045166 16.18 MB/sec execute 246 sec latency 15.975 ms 2024-05-09T07:21:34.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1048436 16.19 MB/sec execute 247 sec latency 25.056 ms 2024-05-09T07:21:35.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1051786 16.20 MB/sec execute 248 sec latency 36.756 ms 2024-05-09T07:21:36.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1055035 16.21 MB/sec execute 249 sec latency 20.738 ms 2024-05-09T07:21:37.972 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1057553 16.18 MB/sec execute 250 sec latency 21.789 ms 2024-05-09T07:21:38.973 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1060208 16.18 MB/sec execute 251 sec latency 36.395 ms 2024-05-09T07:21:39.973 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1062541 16.17 MB/sec execute 252 sec latency 248.396 ms 2024-05-09T07:21:40.973 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1065079 16.16 MB/sec execute 253 sec latency 126.981 ms 2024-05-09T07:21:41.973 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1066494 16.12 MB/sec execute 254 sec latency 45.413 ms 2024-05-09T07:21:42.973 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1069707 16.13 MB/sec execute 255 sec latency 21.568 ms 2024-05-09T07:21:43.973 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1072887 16.14 MB/sec execute 256 sec latency 20.530 ms 2024-05-09T07:21:44.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1075223 16.11 MB/sec execute 257 sec latency 108.951 ms 2024-05-09T07:21:45.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1076914 16.09 MB/sec execute 258 sec latency 509.471 ms 2024-05-09T07:21:46.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1080160 16.10 MB/sec execute 259 sec latency 18.407 ms 2024-05-09T07:21:47.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1083189 16.10 MB/sec execute 260 sec latency 20.346 ms 2024-05-09T07:21:48.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1085850 16.08 MB/sec execute 261 sec latency 21.846 ms 2024-05-09T07:21:49.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1088115 16.07 MB/sec execute 262 sec latency 26.519 ms 2024-05-09T07:21:50.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1091243 16.08 MB/sec execute 263 sec latency 33.341 ms 2024-05-09T07:21:51.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1094477 16.09 MB/sec execute 264 sec latency 28.759 ms 2024-05-09T07:21:52.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1097706 16.09 MB/sec execute 265 sec latency 29.746 ms 2024-05-09T07:21:53.974 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1100521 16.09 MB/sec execute 266 sec latency 32.596 ms 2024-05-09T07:21:54.976 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1103138 16.07 MB/sec execute 267 sec latency 34.166 ms 2024-05-09T07:21:55.976 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1105762 16.07 MB/sec execute 268 sec latency 19.206 ms 2024-05-09T07:21:56.975 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1108854 16.07 MB/sec execute 269 sec latency 17.222 ms 2024-05-09T07:21:57.975 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1112169 16.08 MB/sec execute 270 sec latency 27.443 ms 2024-05-09T07:21:58.975 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1115309 16.09 MB/sec execute 271 sec latency 39.093 ms 2024-05-09T07:21:59.975 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1118306 16.08 MB/sec execute 272 sec latency 25.441 ms 2024-05-09T07:22:00.976 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1121343 16.08 MB/sec execute 273 sec latency 45.070 ms 2024-05-09T07:22:01.976 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1123936 16.08 MB/sec execute 274 sec latency 29.612 ms 2024-05-09T07:22:02.976 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1126688 16.07 MB/sec execute 275 sec latency 29.414 ms 2024-05-09T07:22:03.976 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1129667 16.08 MB/sec execute 276 sec latency 37.457 ms 2024-05-09T07:22:04.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1132462 16.07 MB/sec execute 277 sec latency 37.890 ms 2024-05-09T07:22:05.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1135201 16.06 MB/sec execute 278 sec latency 19.057 ms 2024-05-09T07:22:06.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1135895 16.02 MB/sec execute 279 sec latency 738.877 ms 2024-05-09T07:22:07.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1137320 15.99 MB/sec execute 280 sec latency 764.223 ms 2024-05-09T07:22:08.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1138302 15.95 MB/sec execute 281 sec latency 55.763 ms 2024-05-09T07:22:09.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1141343 15.95 MB/sec execute 282 sec latency 25.423 ms 2024-05-09T07:22:10.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1144460 15.95 MB/sec execute 283 sec latency 25.072 ms 2024-05-09T07:22:11.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1147715 15.97 MB/sec execute 284 sec latency 24.106 ms 2024-05-09T07:22:12.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1151052 15.98 MB/sec execute 285 sec latency 17.791 ms 2024-05-09T07:22:13.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1153815 15.96 MB/sec execute 286 sec latency 90.018 ms 2024-05-09T07:22:14.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1156785 15.96 MB/sec execute 287 sec latency 32.155 ms 2024-05-09T07:22:15.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1159565 15.96 MB/sec execute 288 sec latency 43.390 ms 2024-05-09T07:22:16.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1162653 15.96 MB/sec execute 289 sec latency 19.493 ms 2024-05-09T07:22:17.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1165735 15.97 MB/sec execute 290 sec latency 33.442 ms 2024-05-09T07:22:18.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1168939 15.98 MB/sec execute 291 sec latency 38.438 ms 2024-05-09T07:22:19.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1172227 15.98 MB/sec execute 292 sec latency 16.359 ms 2024-05-09T07:22:20.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1174887 15.97 MB/sec execute 293 sec latency 97.754 ms 2024-05-09T07:22:21.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1178112 15.97 MB/sec execute 294 sec latency 25.507 ms 2024-05-09T07:22:22.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1181547 15.98 MB/sec execute 295 sec latency 21.711 ms 2024-05-09T07:22:23.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1182852 15.96 MB/sec execute 296 sec latency 225.542 ms 2024-05-09T07:22:24.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1184005 15.92 MB/sec execute 297 sec latency 33.852 ms 2024-05-09T07:22:25.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1187445 15.93 MB/sec execute 298 sec latency 21.774 ms 2024-05-09T07:22:26.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1190645 15.93 MB/sec execute 299 sec latency 18.110 ms 2024-05-09T07:22:27.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1194001 15.94 MB/sec execute 300 sec latency 24.961 ms 2024-05-09T07:22:28.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1197344 15.95 MB/sec execute 301 sec latency 12.135 ms 2024-05-09T07:22:29.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1200686 15.96 MB/sec execute 302 sec latency 30.771 ms 2024-05-09T07:22:30.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1203829 15.96 MB/sec execute 303 sec latency 32.420 ms 2024-05-09T07:22:31.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1207000 15.96 MB/sec execute 304 sec latency 32.320 ms 2024-05-09T07:22:32.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1210064 15.96 MB/sec execute 305 sec latency 17.284 ms 2024-05-09T07:22:33.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1212392 15.95 MB/sec execute 306 sec latency 34.809 ms 2024-05-09T07:22:34.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1215328 15.95 MB/sec execute 307 sec latency 35.311 ms 2024-05-09T07:22:35.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1218443 15.96 MB/sec execute 308 sec latency 35.056 ms 2024-05-09T07:22:36.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1221189 15.95 MB/sec execute 309 sec latency 19.147 ms 2024-05-09T07:22:37.977 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1223529 15.94 MB/sec execute 310 sec latency 19.320 ms 2024-05-09T07:22:38.349 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:22:38.352 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:22:38.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1226187 15.93 MB/sec execute 311 sec latency 29.441 ms 2024-05-09T07:22:39.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1229310 15.94 MB/sec execute 312 sec latency 27.646 ms 2024-05-09T07:22:40.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1232588 15.95 MB/sec execute 313 sec latency 40.766 ms 2024-05-09T07:22:41.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1235565 15.94 MB/sec execute 314 sec latency 14.308 ms 2024-05-09T07:22:42.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1238281 15.93 MB/sec execute 315 sec latency 25.781 ms 2024-05-09T07:22:43.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1240989 15.94 MB/sec execute 316 sec latency 24.200 ms 2024-05-09T07:22:44.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1243899 15.93 MB/sec execute 317 sec latency 38.794 ms 2024-05-09T07:22:45.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1247137 15.94 MB/sec execute 318 sec latency 28.398 ms 2024-05-09T07:22:46.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1250420 15.95 MB/sec execute 319 sec latency 39.629 ms 2024-05-09T07:22:47.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1253644 15.96 MB/sec execute 320 sec latency 13.554 ms 2024-05-09T07:22:48.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1255305 15.93 MB/sec execute 321 sec latency 231.009 ms 2024-05-09T07:22:49.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1256579 15.90 MB/sec execute 322 sec latency 766.830 ms 2024-05-09T07:22:50.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1257598 15.87 MB/sec execute 323 sec latency 50.119 ms 2024-05-09T07:22:51.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1260569 15.87 MB/sec execute 324 sec latency 33.549 ms 2024-05-09T07:22:52.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1263521 15.87 MB/sec execute 325 sec latency 25.624 ms 2024-05-09T07:22:53.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1266645 15.87 MB/sec execute 326 sec latency 25.476 ms 2024-05-09T07:22:54.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1269271 15.87 MB/sec execute 327 sec latency 29.185 ms 2024-05-09T07:22:55.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1272262 15.87 MB/sec execute 328 sec latency 27.404 ms 2024-05-09T07:22:56.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1275675 15.87 MB/sec execute 329 sec latency 27.862 ms 2024-05-09T07:22:57.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1279015 15.88 MB/sec execute 330 sec latency 25.896 ms 2024-05-09T07:22:58.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1282412 15.89 MB/sec execute 331 sec latency 47.902 ms 2024-05-09T07:22:59.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1285656 15.90 MB/sec execute 332 sec latency 23.282 ms 2024-05-09T07:23:00.978 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1288745 15.89 MB/sec execute 333 sec latency 28.403 ms 2024-05-09T07:23:01.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1291701 15.89 MB/sec execute 334 sec latency 25.839 ms 2024-05-09T07:23:02.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1295037 15.90 MB/sec execute 335 sec latency 27.280 ms 2024-05-09T07:23:03.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1297406 15.89 MB/sec execute 336 sec latency 87.293 ms 2024-05-09T07:23:04.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1299217 15.87 MB/sec execute 337 sec latency 222.869 ms 2024-05-09T07:23:05.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1300460 15.85 MB/sec execute 338 sec latency 51.743 ms 2024-05-09T07:23:06.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1303737 15.86 MB/sec execute 339 sec latency 31.234 ms 2024-05-09T07:23:07.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1306899 15.86 MB/sec execute 340 sec latency 31.843 ms 2024-05-09T07:23:08.544 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:23:08.547 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:23:08.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1309928 15.86 MB/sec execute 341 sec latency 18.189 ms 2024-05-09T07:23:09.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1313063 15.85 MB/sec execute 342 sec latency 18.228 ms 2024-05-09T07:23:10.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1316495 15.86 MB/sec execute 343 sec latency 21.288 ms 2024-05-09T07:23:11.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1319731 15.87 MB/sec execute 344 sec latency 20.483 ms 2024-05-09T07:23:12.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1322959 15.87 MB/sec execute 345 sec latency 23.502 ms 2024-05-09T07:23:13.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1325869 15.87 MB/sec execute 346 sec latency 19.087 ms 2024-05-09T07:23:14.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1329124 15.88 MB/sec execute 347 sec latency 33.745 ms 2024-05-09T07:23:15.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1332473 15.89 MB/sec execute 348 sec latency 29.733 ms 2024-05-09T07:23:16.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1335809 15.90 MB/sec execute 349 sec latency 39.910 ms 2024-05-09T07:23:17.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1339111 15.90 MB/sec execute 350 sec latency 31.573 ms 2024-05-09T07:23:18.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1342121 15.90 MB/sec execute 351 sec latency 14.292 ms 2024-05-09T07:23:19.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1344830 15.89 MB/sec execute 352 sec latency 29.932 ms 2024-05-09T07:23:20.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1347345 15.89 MB/sec execute 353 sec latency 23.690 ms 2024-05-09T07:23:21.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1350208 15.89 MB/sec execute 354 sec latency 32.325 ms 2024-05-09T07:23:22.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1353393 15.90 MB/sec execute 355 sec latency 44.883 ms 2024-05-09T07:23:23.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1356815 15.90 MB/sec execute 356 sec latency 35.926 ms 2024-05-09T07:23:24.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1358433 15.88 MB/sec execute 357 sec latency 292.154 ms 2024-05-09T07:23:25.979 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1359738 15.85 MB/sec execute 358 sec latency 303.157 ms 2024-05-09T07:23:26.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1361256 15.83 MB/sec execute 359 sec latency 34.021 ms 2024-05-09T07:23:27.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1364424 15.84 MB/sec execute 360 sec latency 17.571 ms 2024-05-09T07:23:28.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1367657 15.84 MB/sec execute 361 sec latency 20.516 ms 2024-05-09T07:23:29.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1370963 15.85 MB/sec execute 362 sec latency 22.239 ms 2024-05-09T07:23:30.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1373935 15.85 MB/sec execute 363 sec latency 21.259 ms 2024-05-09T07:23:31.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1376208 15.88 MB/sec execute 364 sec latency 55.745 ms 2024-05-09T07:23:32.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1379584 15.89 MB/sec execute 365 sec latency 41.982 ms 2024-05-09T07:23:33.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1383017 15.90 MB/sec execute 366 sec latency 12.353 ms 2024-05-09T07:23:34.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1386486 15.90 MB/sec execute 367 sec latency 11.421 ms 2024-05-09T07:23:35.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1389776 15.91 MB/sec execute 368 sec latency 40.028 ms 2024-05-09T07:23:36.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1393058 15.92 MB/sec execute 369 sec latency 12.170 ms 2024-05-09T07:23:37.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1395692 15.91 MB/sec execute 370 sec latency 60.347 ms 2024-05-09T07:23:38.634 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:23:38.637 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:23:38.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1397862 15.90 MB/sec execute 371 sec latency 89.423 ms 2024-05-09T07:23:39.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1400583 15.90 MB/sec execute 372 sec latency 88.894 ms 2024-05-09T07:23:40.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1401466 15.87 MB/sec execute 373 sec latency 574.889 ms 2024-05-09T07:23:41.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1403566 15.86 MB/sec execute 374 sec latency 54.905 ms 2024-05-09T07:23:42.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1406325 15.85 MB/sec execute 375 sec latency 18.746 ms 2024-05-09T07:23:43.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1408280 15.83 MB/sec execute 376 sec latency 44.046 ms 2024-05-09T07:23:44.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1411223 15.84 MB/sec execute 377 sec latency 24.418 ms 2024-05-09T07:23:45.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1414411 15.84 MB/sec execute 378 sec latency 28.748 ms 2024-05-09T07:23:46.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1417295 15.84 MB/sec execute 379 sec latency 24.711 ms 2024-05-09T07:23:47.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1419976 15.83 MB/sec execute 380 sec latency 14.241 ms 2024-05-09T07:23:48.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1422343 15.83 MB/sec execute 381 sec latency 31.943 ms 2024-05-09T07:23:49.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1425443 15.83 MB/sec execute 382 sec latency 39.453 ms 2024-05-09T07:23:50.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1428720 15.84 MB/sec execute 383 sec latency 20.581 ms 2024-05-09T07:23:51.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1432083 15.85 MB/sec execute 384 sec latency 29.521 ms 2024-05-09T07:23:52.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1435172 15.84 MB/sec execute 385 sec latency 45.171 ms 2024-05-09T07:23:53.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1438253 15.84 MB/sec execute 386 sec latency 18.745 ms 2024-05-09T07:23:54.980 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1441249 15.84 MB/sec execute 387 sec latency 29.873 ms 2024-05-09T07:23:55.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1443607 15.84 MB/sec execute 388 sec latency 20.390 ms 2024-05-09T07:23:56.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1446661 15.84 MB/sec execute 389 sec latency 31.737 ms 2024-05-09T07:23:57.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1449808 15.85 MB/sec execute 390 sec latency 27.403 ms 2024-05-09T07:23:58.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1452129 15.83 MB/sec execute 391 sec latency 30.880 ms 2024-05-09T07:23:59.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1454528 15.83 MB/sec execute 392 sec latency 18.931 ms 2024-05-09T07:24:00.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1456115 15.81 MB/sec execute 393 sec latency 500.576 ms 2024-05-09T07:24:01.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1457082 15.78 MB/sec execute 394 sec latency 608.498 ms 2024-05-09T07:24:02.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1459598 15.78 MB/sec execute 395 sec latency 43.138 ms 2024-05-09T07:24:03.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1462586 15.77 MB/sec execute 396 sec latency 19.488 ms 2024-05-09T07:24:04.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1465211 15.77 MB/sec execute 397 sec latency 22.642 ms 2024-05-09T07:24:05.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1468245 15.77 MB/sec execute 398 sec latency 25.151 ms 2024-05-09T07:24:06.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1471534 15.78 MB/sec execute 399 sec latency 36.048 ms 2024-05-09T07:24:07.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1474769 15.79 MB/sec execute 400 sec latency 57.165 ms 2024-05-09T07:24:08.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1477779 15.78 MB/sec execute 401 sec latency 40.082 ms 2024-05-09T07:24:09.039 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:24:09.042 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:24:09.074 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-05-09T07:24:09.068+0000 7f44cf7fa640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42775) UID: 0 2024-05-09T07:24:09.074 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-05-09T07:24:09.068+0000 7fa6c2e0d640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42775) UID: 0 2024-05-09T07:24:09.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1480278 15.77 MB/sec execute 402 sec latency 28.101 ms 2024-05-09T07:24:10.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1482273 15.76 MB/sec execute 403 sec latency 21.641 ms 2024-05-09T07:24:11.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1485273 15.77 MB/sec execute 404 sec latency 37.393 ms 2024-05-09T07:24:12.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1487613 15.75 MB/sec execute 405 sec latency 39.555 ms 2024-05-09T07:24:13.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1489754 15.75 MB/sec execute 406 sec latency 18.345 ms 2024-05-09T07:24:14.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1492546 15.75 MB/sec execute 407 sec latency 51.364 ms 2024-05-09T07:24:15.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1495756 15.75 MB/sec execute 408 sec latency 30.854 ms 2024-05-09T07:24:16.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1498272 15.74 MB/sec execute 409 sec latency 36.542 ms 2024-05-09T07:24:17.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1501111 15.74 MB/sec execute 410 sec latency 14.560 ms 2024-05-09T07:24:18.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1503662 15.74 MB/sec execute 411 sec latency 30.922 ms 2024-05-09T07:24:19.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1506785 15.74 MB/sec execute 412 sec latency 26.883 ms 2024-05-09T07:24:20.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1509601 15.74 MB/sec execute 413 sec latency 89.664 ms 2024-05-09T07:24:21.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1510065 15.71 MB/sec execute 414 sec latency 742.688 ms 2024-05-09T07:24:22.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1511411 15.69 MB/sec execute 415 sec latency 61.461 ms 2024-05-09T07:24:23.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1513862 15.68 MB/sec execute 416 sec latency 47.617 ms 2024-05-09T07:24:24.981 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1517032 15.69 MB/sec execute 417 sec latency 34.312 ms 2024-05-09T07:24:25.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1519440 15.67 MB/sec execute 418 sec latency 34.899 ms 2024-05-09T07:24:26.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1522131 15.68 MB/sec execute 419 sec latency 29.188 ms 2024-05-09T07:24:27.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1524833 15.67 MB/sec execute 420 sec latency 25.458 ms 2024-05-09T07:24:28.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1527990 15.68 MB/sec execute 421 sec latency 19.679 ms 2024-05-09T07:24:29.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1531131 15.68 MB/sec execute 422 sec latency 25.256 ms 2024-05-09T07:24:30.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1534039 15.68 MB/sec execute 423 sec latency 39.008 ms 2024-05-09T07:24:31.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1537018 15.68 MB/sec execute 424 sec latency 24.067 ms 2024-05-09T07:24:32.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1539370 15.67 MB/sec execute 425 sec latency 40.675 ms 2024-05-09T07:24:33.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1542441 15.68 MB/sec execute 426 sec latency 24.950 ms 2024-05-09T07:24:34.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1545740 15.68 MB/sec execute 427 sec latency 11.687 ms 2024-05-09T07:24:35.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1548930 15.69 MB/sec execute 428 sec latency 50.540 ms 2024-05-09T07:24:36.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1551833 15.68 MB/sec execute 429 sec latency 27.696 ms 2024-05-09T07:24:37.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1554744 15.68 MB/sec execute 430 sec latency 41.110 ms 2024-05-09T07:24:38.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1557204 15.68 MB/sec execute 431 sec latency 13.814 ms 2024-05-09T07:24:39.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1560277 15.68 MB/sec execute 432 sec latency 23.105 ms 2024-05-09T07:24:40.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1563657 15.69 MB/sec execute 433 sec latency 20.731 ms 2024-05-09T07:24:41.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1566820 15.69 MB/sec execute 434 sec latency 26.023 ms 2024-05-09T07:24:42.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1569917 15.69 MB/sec execute 435 sec latency 35.714 ms 2024-05-09T07:24:43.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1572545 15.68 MB/sec execute 436 sec latency 81.792 ms 2024-05-09T07:24:44.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1575280 15.69 MB/sec execute 437 sec latency 38.268 ms 2024-05-09T07:24:45.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1578101 15.68 MB/sec execute 438 sec latency 29.085 ms 2024-05-09T07:24:46.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1580229 15.68 MB/sec execute 439 sec latency 301.674 ms 2024-05-09T07:24:47.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1580575 15.64 MB/sec execute 440 sec latency 551.220 ms 2024-05-09T07:24:48.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1581085 15.62 MB/sec execute 441 sec latency 51.713 ms 2024-05-09T07:24:49.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1583946 15.61 MB/sec execute 442 sec latency 45.934 ms 2024-05-09T07:24:50.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1586680 15.61 MB/sec execute 443 sec latency 51.552 ms 2024-05-09T07:24:51.982 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1589083 15.60 MB/sec execute 444 sec latency 26.345 ms 2024-05-09T07:24:52.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1592074 15.61 MB/sec execute 445 sec latency 66.258 ms 2024-05-09T07:24:53.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1595366 15.62 MB/sec execute 446 sec latency 25.180 ms 2024-05-09T07:24:54.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1598170 15.61 MB/sec execute 447 sec latency 28.050 ms 2024-05-09T07:24:55.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1600524 15.60 MB/sec execute 448 sec latency 47.156 ms 2024-05-09T07:24:56.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1603156 15.60 MB/sec execute 449 sec latency 21.806 ms 2024-05-09T07:24:57.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1606303 15.60 MB/sec execute 450 sec latency 32.609 ms 2024-05-09T07:24:58.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1609467 15.61 MB/sec execute 451 sec latency 18.240 ms 2024-05-09T07:24:59.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1612474 15.61 MB/sec execute 452 sec latency 47.172 ms 2024-05-09T07:25:00.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1615105 15.60 MB/sec execute 453 sec latency 38.129 ms 2024-05-09T07:25:01.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1617491 15.60 MB/sec execute 454 sec latency 34.132 ms 2024-05-09T07:25:02.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1620504 15.60 MB/sec execute 455 sec latency 30.568 ms 2024-05-09T07:25:03.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1623702 15.61 MB/sec execute 456 sec latency 21.957 ms 2024-05-09T07:25:04.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1626351 15.60 MB/sec execute 457 sec latency 37.092 ms 2024-05-09T07:25:05.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1628434 15.59 MB/sec execute 458 sec latency 41.395 ms 2024-05-09T07:25:06.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1631187 15.59 MB/sec execute 459 sec latency 23.975 ms 2024-05-09T07:25:07.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1634320 15.60 MB/sec execute 460 sec latency 19.671 ms 2024-05-09T07:25:08.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1635489 15.58 MB/sec execute 461 sec latency 479.734 ms 2024-05-09T07:25:09.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1636921 15.56 MB/sec execute 462 sec latency 684.840 ms 2024-05-09T07:25:10.983 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1638164 15.54 MB/sec execute 463 sec latency 61.298 ms 2024-05-09T07:25:11.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1641333 15.55 MB/sec execute 464 sec latency 32.844 ms 2024-05-09T07:25:12.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1644430 15.54 MB/sec execute 465 sec latency 33.910 ms 2024-05-09T07:25:13.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1647385 15.54 MB/sec execute 466 sec latency 36.628 ms 2024-05-09T07:25:14.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1649887 15.54 MB/sec execute 467 sec latency 35.546 ms 2024-05-09T07:25:15.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1652649 15.54 MB/sec execute 468 sec latency 13.178 ms 2024-05-09T07:25:16.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1655816 15.54 MB/sec execute 469 sec latency 35.542 ms 2024-05-09T07:25:17.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1658934 15.55 MB/sec execute 470 sec latency 25.611 ms 2024-05-09T07:25:18.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1661717 15.54 MB/sec execute 471 sec latency 29.310 ms 2024-05-09T07:25:19.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1664059 15.54 MB/sec execute 472 sec latency 45.494 ms 2024-05-09T07:25:20.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1666897 15.54 MB/sec execute 473 sec latency 34.379 ms 2024-05-09T07:25:21.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1670023 15.54 MB/sec execute 474 sec latency 39.612 ms 2024-05-09T07:25:22.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1673229 15.55 MB/sec execute 475 sec latency 30.407 ms 2024-05-09T07:25:23.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1675720 15.54 MB/sec execute 476 sec latency 27.112 ms 2024-05-09T07:25:24.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1677557 15.53 MB/sec execute 477 sec latency 60.090 ms 2024-05-09T07:25:25.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1680519 15.53 MB/sec execute 478 sec latency 18.794 ms 2024-05-09T07:25:26.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1683263 15.53 MB/sec execute 479 sec latency 27.609 ms 2024-05-09T07:25:27.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1685340 15.52 MB/sec execute 480 sec latency 34.270 ms 2024-05-09T07:25:28.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1688177 15.52 MB/sec execute 481 sec latency 36.880 ms 2024-05-09T07:25:29.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1691469 15.52 MB/sec execute 482 sec latency 17.381 ms 2024-05-09T07:25:30.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1694664 15.53 MB/sec execute 483 sec latency 33.206 ms 2024-05-09T07:25:31.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1697831 15.53 MB/sec execute 484 sec latency 23.449 ms 2024-05-09T07:25:32.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1700622 15.53 MB/sec execute 485 sec latency 34.663 ms 2024-05-09T07:25:33.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1703178 15.52 MB/sec execute 486 sec latency 41.428 ms 2024-05-09T07:25:34.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1705830 15.52 MB/sec execute 487 sec latency 38.936 ms 2024-05-09T07:25:35.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1706405 15.50 MB/sec execute 488 sec latency 721.018 ms 2024-05-09T07:25:36.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1708620 15.49 MB/sec execute 489 sec latency 813.389 ms 2024-05-09T07:25:37.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1709122 15.46 MB/sec execute 490 sec latency 33.153 ms 2024-05-09T07:25:38.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1712010 15.47 MB/sec execute 491 sec latency 35.838 ms 2024-05-09T07:25:39.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1714932 15.46 MB/sec execute 492 sec latency 22.616 ms 2024-05-09T07:25:40.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1717424 15.46 MB/sec execute 493 sec latency 36.550 ms 2024-05-09T07:25:41.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1720023 15.45 MB/sec execute 494 sec latency 29.094 ms 2024-05-09T07:25:42.984 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1723310 15.46 MB/sec execute 495 sec latency 55.909 ms 2024-05-09T07:25:43.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1726513 15.47 MB/sec execute 496 sec latency 35.274 ms 2024-05-09T07:25:44.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1729724 15.47 MB/sec execute 497 sec latency 27.175 ms 2024-05-09T07:25:45.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1732492 15.47 MB/sec execute 498 sec latency 52.272 ms 2024-05-09T07:25:46.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1734967 15.46 MB/sec execute 499 sec latency 33.323 ms 2024-05-09T07:25:47.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1737726 15.46 MB/sec execute 500 sec latency 38.718 ms 2024-05-09T07:25:48.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1740846 15.47 MB/sec execute 501 sec latency 22.465 ms 2024-05-09T07:25:49.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1744013 15.47 MB/sec execute 502 sec latency 19.504 ms 2024-05-09T07:25:50.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1746506 15.46 MB/sec execute 503 sec latency 44.958 ms 2024-05-09T07:25:51.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1748873 15.46 MB/sec execute 504 sec latency 31.194 ms 2024-05-09T07:25:52.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1751816 15.46 MB/sec execute 505 sec latency 19.917 ms 2024-05-09T07:25:53.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1755152 15.46 MB/sec execute 506 sec latency 12.458 ms 2024-05-09T07:25:54.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1758235 15.47 MB/sec execute 507 sec latency 43.936 ms 2024-05-09T07:25:55.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1759662 15.45 MB/sec execute 508 sec latency 282.011 ms 2024-05-09T07:25:56.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1761510 15.44 MB/sec execute 509 sec latency 628.096 ms 2024-05-09T07:25:57.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1762124 15.42 MB/sec execute 510 sec latency 56.122 ms 2024-05-09T07:25:58.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1765245 15.42 MB/sec execute 511 sec latency 34.071 ms 2024-05-09T07:25:59.985 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1768017 15.42 MB/sec execute 512 sec latency 24.850 ms 2024-05-09T07:26:00.987 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1770625 15.41 MB/sec execute 513 sec latency 37.007 ms 2024-05-09T07:26:01.987 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1773122 15.41 MB/sec execute 514 sec latency 21.899 ms 2024-05-09T07:26:02.987 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1776332 15.42 MB/sec execute 515 sec latency 24.287 ms 2024-05-09T07:26:03.987 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1779496 15.42 MB/sec execute 516 sec latency 20.287 ms 2024-05-09T07:26:04.987 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1782041 15.41 MB/sec execute 517 sec latency 38.065 ms 2024-05-09T07:26:05.987 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1784871 15.41 MB/sec execute 518 sec latency 21.456 ms 2024-05-09T07:26:06.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1787437 15.41 MB/sec execute 519 sec latency 36.024 ms 2024-05-09T07:26:07.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1790519 15.42 MB/sec execute 520 sec latency 27.929 ms 2024-05-09T07:26:08.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1793695 15.42 MB/sec execute 521 sec latency 39.166 ms 2024-05-09T07:26:09.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1796010 15.41 MB/sec execute 522 sec latency 44.470 ms 2024-05-09T07:26:10.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1798143 15.40 MB/sec execute 523 sec latency 34.077 ms 2024-05-09T07:26:11.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1801108 15.41 MB/sec execute 524 sec latency 28.925 ms 2024-05-09T07:26:12.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1803704 15.40 MB/sec execute 525 sec latency 19.993 ms 2024-05-09T07:26:13.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1805813 15.40 MB/sec execute 526 sec latency 30.803 ms 2024-05-09T07:26:14.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1808400 15.39 MB/sec execute 527 sec latency 53.234 ms 2024-05-09T07:26:15.989 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1811324 15.40 MB/sec execute 528 sec latency 20.220 ms 2024-05-09T07:26:16.990 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1813485 15.38 MB/sec execute 529 sec latency 48.998 ms 2024-05-09T07:26:17.990 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1815800 15.38 MB/sec execute 530 sec latency 54.929 ms 2024-05-09T07:26:18.990 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1818997 15.38 MB/sec execute 531 sec latency 37.976 ms 2024-05-09T07:26:19.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1822250 15.39 MB/sec execute 532 sec latency 10.940 ms 2024-05-09T07:26:20.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1825165 15.39 MB/sec execute 533 sec latency 23.730 ms 2024-05-09T07:26:21.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1827944 15.38 MB/sec execute 534 sec latency 28.021 ms 2024-05-09T07:26:22.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1830479 15.39 MB/sec execute 535 sec latency 34.549 ms 2024-05-09T07:26:23.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1833376 15.38 MB/sec execute 536 sec latency 91.912 ms 2024-05-09T07:26:24.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1836358 15.42 MB/sec execute 537 sec latency 43.005 ms 2024-05-09T07:26:25.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1839389 15.41 MB/sec execute 538 sec latency 50.726 ms 2024-05-09T07:26:26.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1840756 15.41 MB/sec execute 539 sec latency 315.799 ms 2024-05-09T07:26:27.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1841052 15.38 MB/sec execute 540 sec latency 509.447 ms 2024-05-09T07:26:28.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1841656 15.36 MB/sec execute 541 sec latency 48.583 ms 2024-05-09T07:26:29.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1844181 15.36 MB/sec execute 542 sec latency 45.688 ms 2024-05-09T07:26:30.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1847223 15.35 MB/sec execute 543 sec latency 12.409 ms 2024-05-09T07:26:31.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1849972 15.35 MB/sec execute 544 sec latency 28.498 ms 2024-05-09T07:26:32.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1852337 15.35 MB/sec execute 545 sec latency 22.154 ms 2024-05-09T07:26:33.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1855296 15.35 MB/sec execute 546 sec latency 35.229 ms 2024-05-09T07:26:34.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1858511 15.36 MB/sec execute 547 sec latency 22.727 ms 2024-05-09T07:26:35.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1861229 15.35 MB/sec execute 548 sec latency 37.030 ms 2024-05-09T07:26:36.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1864171 15.35 MB/sec execute 549 sec latency 14.605 ms 2024-05-09T07:26:37.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1866599 15.35 MB/sec execute 550 sec latency 41.844 ms 2024-05-09T07:26:38.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1869606 15.35 MB/sec execute 551 sec latency 43.814 ms 2024-05-09T07:26:39.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1872723 15.36 MB/sec execute 552 sec latency 38.540 ms 2024-05-09T07:26:40.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1875639 15.35 MB/sec execute 553 sec latency 32.870 ms 2024-05-09T07:26:41.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1878369 15.35 MB/sec execute 554 sec latency 20.421 ms 2024-05-09T07:26:42.991 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1880659 15.35 MB/sec execute 555 sec latency 35.100 ms 2024-05-09T07:26:43.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1883621 15.35 MB/sec execute 556 sec latency 37.613 ms 2024-05-09T07:26:44.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1886750 15.35 MB/sec execute 557 sec latency 19.528 ms 2024-05-09T07:26:45.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1889221 15.34 MB/sec execute 558 sec latency 30.754 ms 2024-05-09T07:26:46.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1892150 15.35 MB/sec execute 559 sec latency 12.542 ms 2024-05-09T07:26:47.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1894505 15.34 MB/sec execute 560 sec latency 82.908 ms 2024-05-09T07:26:48.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1895594 15.33 MB/sec execute 561 sec latency 626.513 ms 2024-05-09T07:26:49.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1897634 15.32 MB/sec execute 562 sec latency 729.090 ms 2024-05-09T07:26:50.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1898349 15.30 MB/sec execute 563 sec latency 70.422 ms 2024-05-09T07:26:51.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1901463 15.30 MB/sec execute 564 sec latency 38.066 ms 2024-05-09T07:26:52.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1904609 15.31 MB/sec execute 565 sec latency 21.015 ms 2024-05-09T07:26:53.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1907338 15.30 MB/sec execute 566 sec latency 39.800 ms 2024-05-09T07:26:54.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1909557 15.30 MB/sec execute 567 sec latency 25.409 ms 2024-05-09T07:26:55.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1912217 15.30 MB/sec execute 568 sec latency 62.519 ms 2024-05-09T07:26:56.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1915385 15.30 MB/sec execute 569 sec latency 25.337 ms 2024-05-09T07:26:57.379 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:26:57.382 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:26:57.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1918687 15.31 MB/sec execute 570 sec latency 18.906 ms 2024-05-09T07:26:58.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1921533 15.30 MB/sec execute 571 sec latency 33.194 ms 2024-05-09T07:26:59.994 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1923878 15.30 MB/sec execute 572 sec latency 35.883 ms 2024-05-09T07:27:00.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1926704 15.30 MB/sec execute 573 sec latency 29.249 ms 2024-05-09T07:27:01.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1929794 15.30 MB/sec execute 574 sec latency 31.999 ms 2024-05-09T07:27:02.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1932810 15.31 MB/sec execute 575 sec latency 29.136 ms 2024-05-09T07:27:03.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1935502 15.30 MB/sec execute 576 sec latency 27.537 ms 2024-05-09T07:27:04.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1937926 15.30 MB/sec execute 577 sec latency 27.531 ms 2024-05-09T07:27:05.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1940578 15.30 MB/sec execute 578 sec latency 38.925 ms 2024-05-09T07:27:06.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1943737 15.30 MB/sec execute 579 sec latency 28.866 ms 2024-05-09T07:27:07.992 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1946918 15.31 MB/sec execute 580 sec latency 29.898 ms 2024-05-09T07:27:08.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1949676 15.30 MB/sec execute 581 sec latency 31.933 ms 2024-05-09T07:27:09.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1952292 15.30 MB/sec execute 582 sec latency 22.839 ms 2024-05-09T07:27:10.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1954810 15.30 MB/sec execute 583 sec latency 30.326 ms 2024-05-09T07:27:11.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1958135 15.30 MB/sec execute 584 sec latency 34.407 ms 2024-05-09T07:27:12.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1961327 15.31 MB/sec execute 585 sec latency 48.216 ms 2024-05-09T07:27:13.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1964271 15.30 MB/sec execute 586 sec latency 19.792 ms 2024-05-09T07:27:14.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1964291 15.28 MB/sec execute 587 sec latency 602.293 ms 2024-05-09T07:27:15.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1965115 15.26 MB/sec execute 588 sec latency 271.310 ms 2024-05-09T07:27:16.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1967419 15.26 MB/sec execute 589 sec latency 40.577 ms 2024-05-09T07:27:17.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1969338 15.25 MB/sec execute 590 sec latency 94.723 ms 2024-05-09T07:27:18.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1972524 15.25 MB/sec execute 591 sec latency 12.071 ms 2024-05-09T07:27:19.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1975608 15.26 MB/sec execute 592 sec latency 35.804 ms 2024-05-09T07:27:20.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1978909 15.26 MB/sec execute 593 sec latency 25.334 ms 2024-05-09T07:27:21.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1981815 15.26 MB/sec execute 594 sec latency 34.339 ms 2024-05-09T07:27:22.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1984784 15.26 MB/sec execute 595 sec latency 38.202 ms 2024-05-09T07:27:23.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1987535 15.26 MB/sec execute 596 sec latency 13.431 ms 2024-05-09T07:27:24.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1990208 15.26 MB/sec execute 597 sec latency 26.079 ms 2024-05-09T07:27:25.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1993485 15.26 MB/sec execute 598 sec latency 58.220 ms 2024-05-09T07:27:26.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 1996706 15.27 MB/sec execute 599 sec latency 32.446 ms 2024-05-09T07:27:27.433 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:27:27.436 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:27:27.993 INFO:tasks.workunit.client.1.smithi188.stdout: 1 cleanup 600 sec 2024-05-09T07:27:28.110 INFO:tasks.workunit.client.1.smithi188.stdout: 0 cleanup 600 sec 2024-05-09T07:27:28.110 INFO:tasks.workunit.client.1.smithi188.stdout: 2024-05-09T07:27:28.110 INFO:tasks.workunit.client.1.smithi188.stdout: Operation Count AvgLat MaxLat 2024-05-09T07:27:28.110 INFO:tasks.workunit.client.1.smithi188.stdout: ---------------------------------------- 2024-05-09T07:27:28.110 INFO:tasks.workunit.client.1.smithi188.stdout: NTCreateX 291463 0.709 781.490 2024-05-09T07:27:28.110 INFO:tasks.workunit.client.1.smithi188.stdout: Close 214132 0.002 0.052 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Rename 12343 3.378 551.209 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Unlink 58825 1.189 158.182 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Deltree 8 71.951 209.610 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Mkdir 4 0.007 0.008 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Qpathinfo 264207 0.530 616.596 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Qfileinfo 46337 0.001 0.019 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Qfsinfo 48417 0.131 7.041 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Sfileinfo 23744 0.014 4.625 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Find 102130 0.058 43.926 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: WriteX 145444 0.035 766.822 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: ReadX 456891 0.020 764.216 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: LockX 950 0.996 277.788 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: UnlockX 950 0.622 60.256 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: Flush 20420 5.352 813.379 2024-05-09T07:27:28.111 INFO:tasks.workunit.client.1.smithi188.stdout: 2024-05-09T07:27:28.112 INFO:tasks.workunit.client.1.smithi188.stdout:Throughput 15.2677 MB/sec 1 clients 1 procs max_latency=813.389 ms 2024-05-09T07:27:28.263 INFO:tasks.workunit.client.1.smithi188.stderr:+ dbench 10 2024-05-09T07:27:28.265 INFO:tasks.workunit.client.1.smithi188.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-05-09T07:27:28.265 INFO:tasks.workunit.client.1.smithi188.stdout: 2024-05-09T07:27:28.265 INFO:tasks.workunit.client.1.smithi188.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-05-09T07:27:28.265 INFO:tasks.workunit.client.1.smithi188.stdout:2 of 10 processes prepared for launch 0 sec 2024-05-09T07:27:28.365 INFO:tasks.workunit.client.1.smithi188.stdout:10 of 10 processes prepared for launch 0 sec 2024-05-09T07:27:28.365 INFO:tasks.workunit.client.1.smithi188.stdout:releasing clients 2024-05-09T07:27:29.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 471 165.38 MB/sec warmup 1 sec latency 76.148 ms 2024-05-09T07:27:30.365 INFO:tasks.workunit.client.1.smithi188.stdout: 10 533 91.86 MB/sec warmup 2 sec latency 618.713 ms 2024-05-09T07:27:31.365 INFO:tasks.workunit.client.1.smithi188.stdout: 10 821 71.66 MB/sec warmup 3 sec latency 437.046 ms 2024-05-09T07:27:32.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 1297 60.63 MB/sec warmup 4 sec latency 81.279 ms 2024-05-09T07:27:33.365 INFO:tasks.workunit.client.1.smithi188.stdout: 10 2349 55.76 MB/sec warmup 5 sec latency 37.569 ms 2024-05-09T07:27:34.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 2978 55.83 MB/sec warmup 6 sec latency 37.415 ms 2024-05-09T07:27:35.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 3637 55.18 MB/sec warmup 7 sec latency 65.334 ms 2024-05-09T07:27:36.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 3913 50.01 MB/sec warmup 8 sec latency 60.389 ms 2024-05-09T07:27:37.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 4127 44.55 MB/sec warmup 9 sec latency 70.335 ms 2024-05-09T07:27:38.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 4353 40.54 MB/sec warmup 10 sec latency 217.611 ms 2024-05-09T07:27:39.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 4926 40.39 MB/sec warmup 11 sec latency 66.587 ms 2024-05-09T07:27:40.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 5862 39.07 MB/sec warmup 12 sec latency 50.029 ms 2024-05-09T07:27:41.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 6489 40.13 MB/sec warmup 13 sec latency 35.156 ms 2024-05-09T07:27:42.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 7150 41.01 MB/sec warmup 14 sec latency 58.575 ms 2024-05-09T07:27:43.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 7406 39.29 MB/sec warmup 15 sec latency 116.323 ms 2024-05-09T07:27:44.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 7612 37.00 MB/sec warmup 16 sec latency 74.441 ms 2024-05-09T07:27:45.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 7824 34.99 MB/sec warmup 17 sec latency 164.479 ms 2024-05-09T07:27:46.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 8298 34.14 MB/sec warmup 18 sec latency 86.491 ms 2024-05-09T07:27:47.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 9240 34.26 MB/sec warmup 19 sec latency 40.481 ms 2024-05-09T07:27:48.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 9985 35.41 MB/sec warmup 20 sec latency 28.133 ms 2024-05-09T07:27:49.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 10668 36.41 MB/sec warmup 21 sec latency 36.460 ms 2024-05-09T07:27:50.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 10933 35.47 MB/sec warmup 22 sec latency 73.675 ms 2024-05-09T07:27:51.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 11150 34.05 MB/sec warmup 23 sec latency 64.757 ms 2024-05-09T07:27:52.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 11345 32.73 MB/sec warmup 24 sec latency 188.521 ms 2024-05-09T07:27:53.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 11803 32.17 MB/sec warmup 25 sec latency 83.843 ms 2024-05-09T07:27:54.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 12608 32.29 MB/sec warmup 26 sec latency 47.367 ms 2024-05-09T07:27:55.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 13325 32.52 MB/sec warmup 27 sec latency 35.553 ms 2024-05-09T07:27:56.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 14003 33.33 MB/sec warmup 28 sec latency 73.679 ms 2024-05-09T07:27:57.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 14417 33.41 MB/sec warmup 29 sec latency 79.970 ms 2024-05-09T07:27:57.508 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:27:57.512 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:27:58.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 14640 32.47 MB/sec warmup 30 sec latency 61.809 ms 2024-05-09T07:27:59.366 INFO:tasks.workunit.client.1.smithi188.stdout: 10 14876 31.51 MB/sec warmup 31 sec latency 66.391 ms 2024-05-09T07:28:00.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 15063 30.67 MB/sec warmup 32 sec latency 200.190 ms 2024-05-09T07:28:01.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 15075 29.75 MB/sec warmup 33 sec latency 1163.774 ms 2024-05-09T07:28:02.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 15086 28.88 MB/sec warmup 34 sec latency 2030.105 ms 2024-05-09T07:28:03.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 15170 28.23 MB/sec warmup 35 sec latency 2300.278 ms 2024-05-09T07:28:04.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 15346 27.65 MB/sec warmup 36 sec latency 190.017 ms 2024-05-09T07:28:05.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 15686 27.57 MB/sec warmup 37 sec latency 163.123 ms 2024-05-09T07:28:06.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 16195 27.20 MB/sec warmup 38 sec latency 96.752 ms 2024-05-09T07:28:07.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 16917 27.51 MB/sec warmup 39 sec latency 33.898 ms 2024-05-09T07:28:08.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 17590 28.29 MB/sec warmup 40 sec latency 85.783 ms 2024-05-09T07:28:09.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 17955 28.30 MB/sec warmup 41 sec latency 74.230 ms 2024-05-09T07:28:10.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 18170 27.74 MB/sec warmup 42 sec latency 41.815 ms 2024-05-09T07:28:11.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 18365 27.14 MB/sec warmup 43 sec latency 131.779 ms 2024-05-09T07:28:12.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 18690 26.77 MB/sec warmup 44 sec latency 95.883 ms 2024-05-09T07:28:13.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 19332 26.93 MB/sec warmup 45 sec latency 62.025 ms 2024-05-09T07:28:14.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 20201 27.02 MB/sec warmup 46 sec latency 42.648 ms 2024-05-09T07:28:15.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 20907 27.62 MB/sec warmup 47 sec latency 96.262 ms 2024-05-09T07:28:16.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 21175 27.48 MB/sec warmup 48 sec latency 432.440 ms 2024-05-09T07:28:17.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 21281 27.11 MB/sec warmup 49 sec latency 695.966 ms 2024-05-09T07:28:18.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 21524 26.94 MB/sec warmup 50 sec latency 123.713 ms 2024-05-09T07:28:19.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 21704 26.57 MB/sec warmup 51 sec latency 80.119 ms 2024-05-09T07:28:20.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 21902 26.12 MB/sec warmup 52 sec latency 107.786 ms 2024-05-09T07:28:21.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 22166 25.76 MB/sec warmup 53 sec latency 125.868 ms 2024-05-09T07:28:22.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 22617 25.76 MB/sec warmup 54 sec latency 140.458 ms 2024-05-09T07:28:23.367 INFO:tasks.workunit.client.1.smithi188.stdout: 10 23296 25.71 MB/sec warmup 55 sec latency 123.478 ms 2024-05-09T07:28:24.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 23918 25.92 MB/sec warmup 56 sec latency 72.298 ms 2024-05-09T07:28:25.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 24555 26.24 MB/sec warmup 57 sec latency 90.912 ms 2024-05-09T07:28:26.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 25015 26.49 MB/sec warmup 58 sec latency 89.470 ms 2024-05-09T07:28:27.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 25257 26.21 MB/sec warmup 59 sec latency 86.687 ms 2024-05-09T07:28:27.605 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:28:27.608 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:28:27.646 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-05-09T07:28:27.638+0000 7f44cf7fa640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42870) UID: 0 2024-05-09T07:28:27.646 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-05-09T07:28:27.638+0000 7fa6c2e0d640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42870) UID: 0 2024-05-09T07:28:28.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 25449 25.82 MB/sec warmup 60 sec latency 113.319 ms 2024-05-09T07:28:29.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 25738 25.53 MB/sec warmup 61 sec latency 100.652 ms 2024-05-09T07:28:30.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 26132 25.42 MB/sec warmup 62 sec latency 160.531 ms 2024-05-09T07:28:31.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 26869 25.47 MB/sec warmup 63 sec latency 103.533 ms 2024-05-09T07:28:32.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 26903 25.12 MB/sec warmup 64 sec latency 1094.871 ms 2024-05-09T07:28:33.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 27217 24.96 MB/sec warmup 65 sec latency 1136.443 ms 2024-05-09T07:28:34.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 27587 24.94 MB/sec warmup 66 sec latency 98.933 ms 2024-05-09T07:28:35.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 28219 25.40 MB/sec warmup 67 sec latency 36.480 ms 2024-05-09T07:28:36.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 28579 25.43 MB/sec warmup 68 sec latency 108.826 ms 2024-05-09T07:28:37.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 28789 25.20 MB/sec warmup 69 sec latency 75.119 ms 2024-05-09T07:28:38.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 28971 24.87 MB/sec warmup 70 sec latency 92.062 ms 2024-05-09T07:28:39.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 29191 24.58 MB/sec warmup 71 sec latency 112.657 ms 2024-05-09T07:28:40.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 29590 24.51 MB/sec warmup 72 sec latency 124.771 ms 2024-05-09T07:28:41.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 30301 24.55 MB/sec warmup 73 sec latency 66.472 ms 2024-05-09T07:28:42.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 31089 24.81 MB/sec warmup 74 sec latency 49.625 ms 2024-05-09T07:28:43.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 31784 25.25 MB/sec warmup 75 sec latency 33.992 ms 2024-05-09T07:28:44.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 32082 25.21 MB/sec warmup 76 sec latency 223.032 ms 2024-05-09T07:28:45.368 INFO:tasks.workunit.client.1.smithi188.stdout: 10 32300 25.03 MB/sec warmup 77 sec latency 183.028 ms 2024-05-09T07:28:46.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 32475 24.75 MB/sec warmup 78 sec latency 174.837 ms 2024-05-09T07:28:47.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 32559 24.46 MB/sec warmup 79 sec latency 698.518 ms 2024-05-09T07:28:48.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 32613 24.17 MB/sec warmup 80 sec latency 1174.259 ms 2024-05-09T07:28:49.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 32863 23.98 MB/sec warmup 81 sec latency 132.921 ms 2024-05-09T07:28:50.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 33131 23.92 MB/sec warmup 82 sec latency 108.408 ms 2024-05-09T07:28:51.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 33674 23.83 MB/sec warmup 83 sec latency 90.266 ms 2024-05-09T07:28:52.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 34405 23.97 MB/sec warmup 84 sec latency 51.543 ms 2024-05-09T07:28:53.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 35148 24.34 MB/sec warmup 85 sec latency 40.054 ms 2024-05-09T07:28:54.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 35547 24.45 MB/sec warmup 86 sec latency 203.972 ms 2024-05-09T07:28:55.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 35816 24.35 MB/sec warmup 87 sec latency 88.134 ms 2024-05-09T07:28:56.369 INFO:tasks.workunit.client.1.smithi188.stdout: 10 36011 24.11 MB/sec warmup 88 sec latency 110.026 ms 2024-05-09T07:28:57.370 INFO:tasks.workunit.client.1.smithi188.stdout: 10 36187 23.87 MB/sec warmup 89 sec latency 117.763 ms 2024-05-09T07:28:58.370 INFO:tasks.workunit.client.1.smithi188.stdout: 10 36571 23.76 MB/sec warmup 90 sec latency 99.518 ms 2024-05-09T07:28:59.371 INFO:tasks.workunit.client.1.smithi188.stdout: 10 37303 23.87 MB/sec warmup 91 sec latency 55.297 ms 2024-05-09T07:29:00.370 INFO:tasks.workunit.client.1.smithi188.stdout: 10 38010 23.99 MB/sec warmup 92 sec latency 37.618 ms 2024-05-09T07:29:01.371 INFO:tasks.workunit.client.1.smithi188.stdout: 10 38626 24.21 MB/sec warmup 93 sec latency 107.929 ms 2024-05-09T07:29:02.371 INFO:tasks.workunit.client.1.smithi188.stdout: 10 38964 24.30 MB/sec warmup 94 sec latency 383.835 ms 2024-05-09T07:29:03.371 INFO:tasks.workunit.client.1.smithi188.stdout: 10 39007 24.08 MB/sec warmup 95 sec latency 1033.914 ms 2024-05-09T07:29:04.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 39230 23.98 MB/sec warmup 96 sec latency 1149.305 ms 2024-05-09T07:29:05.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 39368 23.82 MB/sec warmup 97 sec latency 76.666 ms 2024-05-09T07:29:06.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 39543 23.60 MB/sec warmup 98 sec latency 158.038 ms 2024-05-09T07:29:07.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 39750 23.40 MB/sec warmup 99 sec latency 148.615 ms 2024-05-09T07:29:08.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 40044 23.26 MB/sec warmup 100 sec latency 156.179 ms 2024-05-09T07:29:09.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 40595 23.34 MB/sec warmup 101 sec latency 97.653 ms 2024-05-09T07:29:10.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 41338 23.34 MB/sec warmup 102 sec latency 47.156 ms 2024-05-09T07:29:11.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 42051 23.63 MB/sec warmup 103 sec latency 36.421 ms 2024-05-09T07:29:12.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 42504 23.76 MB/sec warmup 104 sec latency 89.251 ms 2024-05-09T07:29:13.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 42831 23.77 MB/sec warmup 105 sec latency 61.828 ms 2024-05-09T07:29:14.375 INFO:tasks.workunit.client.1.smithi188.stdout: 10 43047 23.61 MB/sec warmup 106 sec latency 107.794 ms 2024-05-09T07:29:15.374 INFO:tasks.workunit.client.1.smithi188.stdout: 10 43228 23.42 MB/sec warmup 107 sec latency 124.391 ms 2024-05-09T07:29:16.373 INFO:tasks.workunit.client.1.smithi188.stdout: 10 43543 23.30 MB/sec warmup 108 sec latency 108.558 ms 2024-05-09T07:29:17.374 INFO:tasks.workunit.client.1.smithi188.stdout: 10 44023 23.32 MB/sec warmup 109 sec latency 113.136 ms 2024-05-09T07:29:18.374 INFO:tasks.workunit.client.1.smithi188.stdout: 10 44735 23.33 MB/sec warmup 110 sec latency 46.048 ms 2024-05-09T07:29:19.374 INFO:tasks.workunit.client.1.smithi188.stdout: 10 45387 23.53 MB/sec warmup 111 sec latency 41.985 ms 2024-05-09T07:29:20.374 INFO:tasks.workunit.client.1.smithi188.stdout: 10 45959 23.73 MB/sec warmup 112 sec latency 85.866 ms 2024-05-09T07:29:21.374 INFO:tasks.workunit.client.1.smithi188.stdout: 10 46313 23.75 MB/sec warmup 113 sec latency 70.103 ms 2024-05-09T07:29:22.375 INFO:tasks.workunit.client.1.smithi188.stdout: 10 46546 23.63 MB/sec warmup 114 sec latency 127.323 ms 2024-05-09T07:29:23.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 46580 23.42 MB/sec warmup 115 sec latency 908.721 ms 2024-05-09T07:29:24.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 46599 23.22 MB/sec warmup 116 sec latency 1406.937 ms 2024-05-09T07:29:25.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 46736 23.05 MB/sec warmup 117 sec latency 483.771 ms 2024-05-09T07:29:26.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 46886 22.89 MB/sec warmup 118 sec latency 132.314 ms 2024-05-09T07:29:27.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 47075 22.74 MB/sec warmup 119 sec latency 215.423 ms 2024-05-09T07:29:29.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 48140 19.02 MB/sec execute 1 sec latency 149.745 ms 2024-05-09T07:29:30.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 48791 30.93 MB/sec execute 2 sec latency 50.905 ms 2024-05-09T07:29:31.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 49379 34.58 MB/sec execute 3 sec latency 81.423 ms 2024-05-09T07:29:32.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 49815 35.00 MB/sec execute 4 sec latency 100.761 ms 2024-05-09T07:29:33.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 50050 29.69 MB/sec execute 5 sec latency 80.241 ms 2024-05-09T07:29:34.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 50245 25.65 MB/sec execute 6 sec latency 96.483 ms 2024-05-09T07:29:35.380 INFO:tasks.workunit.client.1.smithi188.stdout: 10 50483 23.02 MB/sec execute 7 sec latency 137.758 ms 2024-05-09T07:29:36.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 50899 22.41 MB/sec execute 8 sec latency 56.070 ms 2024-05-09T07:29:37.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 51558 23.08 MB/sec execute 9 sec latency 52.454 ms 2024-05-09T07:29:38.377 INFO:tasks.workunit.client.1.smithi188.stdout: 10 51884 22.49 MB/sec execute 10 sec latency 549.775 ms 2024-05-09T07:29:39.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 52042 21.05 MB/sec execute 11 sec latency 1062.926 ms 2024-05-09T07:29:40.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 52221 20.13 MB/sec execute 12 sec latency 162.249 ms 2024-05-09T07:29:41.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 52646 21.50 MB/sec execute 13 sec latency 158.102 ms 2024-05-09T07:29:42.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 53095 22.17 MB/sec execute 14 sec latency 131.116 ms 2024-05-09T07:29:43.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 53410 22.17 MB/sec execute 15 sec latency 137.194 ms 2024-05-09T07:29:44.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 53641 21.26 MB/sec execute 16 sec latency 74.917 ms 2024-05-09T07:29:45.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 53859 20.42 MB/sec execute 17 sec latency 118.877 ms 2024-05-09T07:29:46.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 54146 19.90 MB/sec execute 18 sec latency 143.758 ms 2024-05-09T07:29:47.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 54691 20.11 MB/sec execute 19 sec latency 117.140 ms 2024-05-09T07:29:48.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 55319 20.47 MB/sec execute 20 sec latency 66.371 ms 2024-05-09T07:29:49.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 56049 21.83 MB/sec execute 21 sec latency 39.406 ms 2024-05-09T07:29:50.380 INFO:tasks.workunit.client.1.smithi188.stdout: 10 56609 22.69 MB/sec execute 22 sec latency 50.337 ms 2024-05-09T07:29:51.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 56963 22.74 MB/sec execute 23 sec latency 82.741 ms 2024-05-09T07:29:52.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 57195 22.13 MB/sec execute 24 sec latency 111.985 ms 2024-05-09T07:29:53.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 57455 21.58 MB/sec execute 25 sec latency 121.811 ms 2024-05-09T07:29:54.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 57866 21.54 MB/sec execute 26 sec latency 85.879 ms 2024-05-09T07:29:55.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 58465 21.43 MB/sec execute 27 sec latency 125.475 ms 2024-05-09T07:29:56.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 59060 22.10 MB/sec execute 28 sec latency 56.268 ms 2024-05-09T07:29:57.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 59694 22.70 MB/sec execute 29 sec latency 115.495 ms 2024-05-09T07:29:58.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 60186 23.06 MB/sec execute 30 sec latency 55.471 ms 2024-05-09T07:29:59.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 60533 23.05 MB/sec execute 31 sec latency 103.396 ms 2024-05-09T07:30:00.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 60759 22.59 MB/sec execute 32 sec latency 103.298 ms 2024-05-09T07:30:01.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 61034 22.24 MB/sec execute 33 sec latency 93.825 ms 2024-05-09T07:30:02.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 61448 22.13 MB/sec execute 34 sec latency 88.374 ms 2024-05-09T07:30:03.378 INFO:tasks.workunit.client.1.smithi188.stdout: 10 61909 21.93 MB/sec execute 35 sec latency 108.606 ms 2024-05-09T07:30:04.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 61972 21.37 MB/sec execute 36 sec latency 883.058 ms 2024-05-09T07:30:05.379 INFO:tasks.workunit.client.1.smithi188.stdout: 10 61999 20.87 MB/sec execute 37 sec latency 1392.103 ms 2024-05-09T07:30:06.382 INFO:tasks.workunit.client.1.smithi188.stdout: 10 62118 20.46 MB/sec execute 38 sec latency 687.200 ms 2024-05-09T07:30:07.382 INFO:tasks.workunit.client.1.smithi188.stdout: 10 62462 20.52 MB/sec execute 39 sec latency 152.969 ms 2024-05-09T07:30:08.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 62793 20.50 MB/sec execute 40 sec latency 106.063 ms 2024-05-09T07:30:09.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 63444 20.98 MB/sec execute 41 sec latency 117.363 ms 2024-05-09T07:30:10.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 63855 21.15 MB/sec execute 42 sec latency 77.499 ms 2024-05-09T07:30:11.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 64197 21.14 MB/sec execute 43 sec latency 80.353 ms 2024-05-09T07:30:12.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 64420 20.89 MB/sec execute 44 sec latency 143.777 ms 2024-05-09T07:30:13.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 64719 20.68 MB/sec execute 45 sec latency 115.648 ms 2024-05-09T07:30:14.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 65138 20.64 MB/sec execute 46 sec latency 94.048 ms 2024-05-09T07:30:15.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 65663 20.61 MB/sec execute 47 sec latency 103.465 ms 2024-05-09T07:30:16.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 66277 21.03 MB/sec execute 48 sec latency 62.292 ms 2024-05-09T07:30:17.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 66892 21.33 MB/sec execute 49 sec latency 56.063 ms 2024-05-09T07:30:18.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 67345 21.53 MB/sec execute 50 sec latency 103.597 ms 2024-05-09T07:30:19.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 67479 21.21 MB/sec execute 51 sec latency 568.311 ms 2024-05-09T07:30:20.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 67533 20.91 MB/sec execute 52 sec latency 945.706 ms 2024-05-09T07:30:21.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 67589 20.58 MB/sec execute 53 sec latency 634.793 ms 2024-05-09T07:30:22.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 67739 20.36 MB/sec execute 54 sec latency 220.407 ms 2024-05-09T07:30:23.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 67904 20.12 MB/sec execute 55 sec latency 232.049 ms 2024-05-09T07:30:24.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 68214 20.01 MB/sec execute 56 sec latency 123.120 ms 2024-05-09T07:30:25.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 68625 19.92 MB/sec execute 57 sec latency 98.162 ms 2024-05-09T07:30:26.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 69067 19.83 MB/sec execute 58 sec latency 103.670 ms 2024-05-09T07:30:27.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 69681 20.20 MB/sec execute 59 sec latency 81.055 ms 2024-05-09T07:30:28.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 70319 20.50 MB/sec execute 60 sec latency 81.344 ms 2024-05-09T07:30:29.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 70734 20.63 MB/sec execute 61 sec latency 92.930 ms 2024-05-09T07:30:30.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 71130 20.72 MB/sec execute 62 sec latency 86.596 ms 2024-05-09T07:30:31.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 71387 20.57 MB/sec execute 63 sec latency 164.619 ms 2024-05-09T07:30:32.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 71732 20.49 MB/sec execute 64 sec latency 58.525 ms 2024-05-09T07:30:33.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 72139 20.39 MB/sec execute 65 sec latency 86.496 ms 2024-05-09T07:30:34.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 72587 20.32 MB/sec execute 66 sec latency 98.349 ms 2024-05-09T07:30:35.383 INFO:tasks.workunit.client.1.smithi188.stdout: 10 73243 20.67 MB/sec execute 67 sec latency 61.015 ms 2024-05-09T07:30:36.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 73885 20.89 MB/sec execute 68 sec latency 50.330 ms 2024-05-09T07:30:37.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 74223 20.93 MB/sec execute 69 sec latency 166.631 ms 2024-05-09T07:30:38.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 74586 20.93 MB/sec execute 70 sec latency 100.320 ms 2024-05-09T07:30:39.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 74826 20.85 MB/sec execute 71 sec latency 146.018 ms 2024-05-09T07:30:40.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 74837 20.57 MB/sec execute 72 sec latency 1113.578 ms 2024-05-09T07:30:41.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 74897 20.31 MB/sec execute 73 sec latency 1526.002 ms 2024-05-09T07:30:42.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 75041 20.13 MB/sec execute 74 sec latency 167.036 ms 2024-05-09T07:30:43.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 75204 19.94 MB/sec execute 75 sec latency 139.554 ms 2024-05-09T07:30:44.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 75509 19.92 MB/sec execute 76 sec latency 117.120 ms 2024-05-09T07:30:45.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 76003 19.85 MB/sec execute 77 sec latency 122.486 ms 2024-05-09T07:30:46.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 76372 19.82 MB/sec execute 78 sec latency 113.706 ms 2024-05-09T07:30:47.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 76995 20.10 MB/sec execute 79 sec latency 56.753 ms 2024-05-09T07:30:48.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 77564 20.27 MB/sec execute 80 sec latency 86.403 ms 2024-05-09T07:30:49.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 77995 20.36 MB/sec execute 81 sec latency 84.073 ms 2024-05-09T07:30:50.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 78335 20.34 MB/sec execute 82 sec latency 97.538 ms 2024-05-09T07:30:51.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 78677 20.28 MB/sec execute 83 sec latency 75.750 ms 2024-05-09T07:30:52.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 79094 20.31 MB/sec execute 84 sec latency 78.004 ms 2024-05-09T07:30:53.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 79607 20.24 MB/sec execute 85 sec latency 102.176 ms 2024-05-09T07:30:54.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 80099 20.35 MB/sec execute 86 sec latency 68.714 ms 2024-05-09T07:30:55.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 80689 20.52 MB/sec execute 87 sec latency 81.976 ms 2024-05-09T07:30:56.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 81210 20.69 MB/sec execute 88 sec latency 81.118 ms 2024-05-09T07:30:57.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 81627 20.72 MB/sec execute 89 sec latency 81.008 ms 2024-05-09T07:30:58.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 81927 20.68 MB/sec execute 90 sec latency 112.679 ms 2024-05-09T07:30:59.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 82233 20.61 MB/sec execute 91 sec latency 137.518 ms 2024-05-09T07:31:00.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 82649 20.59 MB/sec execute 92 sec latency 91.006 ms 2024-05-09T07:31:01.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 83086 20.54 MB/sec execute 93 sec latency 72.033 ms 2024-05-09T07:31:02.384 INFO:tasks.workunit.client.1.smithi188.stdout: 10 83634 20.66 MB/sec execute 94 sec latency 71.131 ms 2024-05-09T07:31:03.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 84216 20.82 MB/sec execute 95 sec latency 104.049 ms 2024-05-09T07:31:04.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 84674 20.88 MB/sec execute 96 sec latency 78.826 ms 2024-05-09T07:31:05.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 84876 20.82 MB/sec execute 97 sec latency 414.248 ms 2024-05-09T07:31:06.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 84885 20.62 MB/sec execute 98 sec latency 1292.816 ms 2024-05-09T07:31:07.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 84911 20.42 MB/sec execute 99 sec latency 1497.634 ms 2024-05-09T07:31:08.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 85066 20.30 MB/sec execute 100 sec latency 755.499 ms 2024-05-09T07:31:09.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 85283 20.21 MB/sec execute 101 sec latency 141.874 ms 2024-05-09T07:31:10.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 85498 20.13 MB/sec execute 102 sec latency 131.752 ms 2024-05-09T07:31:11.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 85785 20.07 MB/sec execute 103 sec latency 103.136 ms 2024-05-09T07:31:12.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 86222 20.08 MB/sec execute 104 sec latency 96.091 ms 2024-05-09T07:31:13.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 86717 20.03 MB/sec execute 105 sec latency 86.291 ms 2024-05-09T07:31:14.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 87320 20.19 MB/sec execute 106 sec latency 52.910 ms 2024-05-09T07:31:15.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 87918 20.33 MB/sec execute 107 sec latency 75.364 ms 2024-05-09T07:31:16.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 88375 20.44 MB/sec execute 108 sec latency 92.663 ms 2024-05-09T07:31:17.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 88827 20.48 MB/sec execute 109 sec latency 66.101 ms 2024-05-09T07:31:18.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 89097 20.44 MB/sec execute 110 sec latency 100.843 ms 2024-05-09T07:31:19.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 89467 20.39 MB/sec execute 111 sec latency 92.697 ms 2024-05-09T07:31:20.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 89817 20.38 MB/sec execute 112 sec latency 159.082 ms 2024-05-09T07:31:21.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 89890 20.22 MB/sec execute 113 sec latency 1156.572 ms 2024-05-09T07:31:22.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 90246 20.12 MB/sec execute 114 sec latency 1223.357 ms 2024-05-09T07:31:23.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 90601 20.14 MB/sec execute 115 sec latency 95.607 ms 2024-05-09T07:31:24.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 91192 20.26 MB/sec execute 116 sec latency 102.161 ms 2024-05-09T07:31:25.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 91708 20.36 MB/sec execute 117 sec latency 109.947 ms 2024-05-09T07:31:26.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 92154 20.46 MB/sec execute 118 sec latency 80.418 ms 2024-05-09T07:31:27.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 92510 20.45 MB/sec execute 119 sec latency 75.184 ms 2024-05-09T07:31:28.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 92802 20.38 MB/sec execute 120 sec latency 91.817 ms 2024-05-09T07:31:29.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 93257 20.42 MB/sec execute 121 sec latency 94.509 ms 2024-05-09T07:31:30.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 93776 20.36 MB/sec execute 122 sec latency 104.529 ms 2024-05-09T07:31:31.385 INFO:tasks.workunit.client.1.smithi188.stdout: 10 94314 20.48 MB/sec execute 123 sec latency 65.535 ms 2024-05-09T07:31:32.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 94870 20.57 MB/sec execute 124 sec latency 111.505 ms 2024-05-09T07:31:33.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 95396 20.70 MB/sec execute 125 sec latency 79.361 ms 2024-05-09T07:31:34.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 95765 20.71 MB/sec execute 126 sec latency 133.145 ms 2024-05-09T07:31:35.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 96101 20.68 MB/sec execute 127 sec latency 90.468 ms 2024-05-09T07:31:36.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 96195 20.55 MB/sec execute 128 sec latency 799.286 ms 2024-05-09T07:31:37.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 96242 20.41 MB/sec execute 129 sec latency 1187.188 ms 2024-05-09T07:31:38.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 96508 20.33 MB/sec execute 130 sec latency 291.928 ms 2024-05-09T07:31:39.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 96736 20.29 MB/sec execute 131 sec latency 133.358 ms 2024-05-09T07:31:40.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 97203 20.24 MB/sec execute 132 sec latency 90.943 ms 2024-05-09T07:31:41.050 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:31:41.054 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:31:41.106 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-05-09T07:31:41.098+0000 7f44cf7fa640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42898) UID: 0 2024-05-09T07:31:41.107 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-05-09T07:31:41.098+0000 7fa6c2e0d640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42898) UID: 0 2024-05-09T07:31:41.386 INFO:tasks.workunit.client.1.smithi188.stdout: 10 97702 20.28 MB/sec execute 133 sec latency 67.585 ms 2024-05-09T07:31:42.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 98260 20.39 MB/sec execute 134 sec latency 108.883 ms 2024-05-09T07:31:43.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 98825 20.53 MB/sec execute 135 sec latency 108.870 ms 2024-05-09T07:31:44.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 99199 20.57 MB/sec execute 136 sec latency 66.394 ms 2024-05-09T07:31:45.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 99518 20.51 MB/sec execute 137 sec latency 119.566 ms 2024-05-09T07:31:46.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 99805 20.47 MB/sec execute 138 sec latency 116.307 ms 2024-05-09T07:31:47.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 100125 20.43 MB/sec execute 139 sec latency 98.173 ms 2024-05-09T07:31:48.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 100572 20.43 MB/sec execute 140 sec latency 74.187 ms 2024-05-09T07:31:49.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 101094 20.42 MB/sec execute 141 sec latency 107.728 ms 2024-05-09T07:31:50.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 101701 20.55 MB/sec execute 142 sec latency 43.474 ms 2024-05-09T07:31:51.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 102194 20.59 MB/sec execute 143 sec latency 93.958 ms 2024-05-09T07:31:52.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 102654 20.71 MB/sec execute 144 sec latency 103.178 ms 2024-05-09T07:31:53.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 103037 20.71 MB/sec execute 145 sec latency 90.622 ms 2024-05-09T07:31:54.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 103270 20.64 MB/sec execute 146 sec latency 180.733 ms 2024-05-09T07:31:55.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 103510 20.56 MB/sec execute 147 sec latency 199.722 ms 2024-05-09T07:31:56.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 103890 20.53 MB/sec execute 148 sec latency 77.819 ms 2024-05-09T07:31:57.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 104432 20.55 MB/sec execute 149 sec latency 66.175 ms 2024-05-09T07:31:58.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 105029 20.61 MB/sec execute 150 sec latency 41.979 ms 2024-05-09T07:31:59.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 105657 20.74 MB/sec execute 151 sec latency 92.025 ms 2024-05-09T07:32:00.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 106143 20.85 MB/sec execute 152 sec latency 129.443 ms 2024-05-09T07:32:01.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 106543 20.86 MB/sec execute 153 sec latency 87.753 ms 2024-05-09T07:32:02.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 106819 20.81 MB/sec execute 154 sec latency 93.082 ms 2024-05-09T07:32:03.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 107076 20.74 MB/sec execute 155 sec latency 103.244 ms 2024-05-09T07:32:04.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 107392 20.70 MB/sec execute 156 sec latency 154.076 ms 2024-05-09T07:32:05.387 INFO:tasks.workunit.client.1.smithi188.stdout: 10 107896 20.70 MB/sec execute 157 sec latency 117.697 ms 2024-05-09T07:32:06.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 108465 20.74 MB/sec execute 158 sec latency 86.349 ms 2024-05-09T07:32:07.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 108496 20.62 MB/sec execute 159 sec latency 994.281 ms 2024-05-09T07:32:08.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 108516 20.50 MB/sec execute 160 sec latency 1994.377 ms 2024-05-09T07:32:09.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 108552 20.38 MB/sec execute 161 sec latency 2199.180 ms 2024-05-09T07:32:10.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 108805 20.33 MB/sec execute 162 sec latency 235.930 ms 2024-05-09T07:32:11.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 109080 20.35 MB/sec execute 163 sec latency 119.563 ms 2024-05-09T07:32:12.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 109297 20.29 MB/sec execute 164 sec latency 220.519 ms 2024-05-09T07:32:13.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 109623 20.34 MB/sec execute 165 sec latency 97.061 ms 2024-05-09T07:32:14.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 109987 20.36 MB/sec execute 166 sec latency 123.496 ms 2024-05-09T07:32:15.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 110313 20.34 MB/sec execute 167 sec latency 58.921 ms 2024-05-09T07:32:16.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 110530 20.25 MB/sec execute 168 sec latency 118.438 ms 2024-05-09T07:32:17.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 110833 20.22 MB/sec execute 169 sec latency 210.774 ms 2024-05-09T07:32:18.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 111288 20.21 MB/sec execute 170 sec latency 99.948 ms 2024-05-09T07:32:19.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 111886 20.24 MB/sec execute 171 sec latency 102.857 ms 2024-05-09T07:32:20.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 112477 20.34 MB/sec execute 172 sec latency 106.647 ms 2024-05-09T07:32:21.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 113095 20.48 MB/sec execute 173 sec latency 128.879 ms 2024-05-09T07:32:22.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 113195 20.41 MB/sec execute 174 sec latency 446.960 ms 2024-05-09T07:32:23.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 113336 20.33 MB/sec execute 175 sec latency 665.794 ms 2024-05-09T07:32:24.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 113537 20.28 MB/sec execute 176 sec latency 135.306 ms 2024-05-09T07:32:25.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 113815 20.23 MB/sec execute 177 sec latency 54.580 ms 2024-05-09T07:32:26.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 113991 20.16 MB/sec execute 178 sec latency 179.263 ms 2024-05-09T07:32:27.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 114249 20.10 MB/sec execute 179 sec latency 155.593 ms 2024-05-09T07:32:28.388 INFO:tasks.workunit.client.1.smithi188.stdout: 10 114629 20.10 MB/sec execute 180 sec latency 191.772 ms 2024-05-09T07:32:29.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 115072 20.09 MB/sec execute 181 sec latency 188.388 ms 2024-05-09T07:32:30.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 115605 20.12 MB/sec execute 182 sec latency 102.467 ms 2024-05-09T07:32:31.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 116155 20.21 MB/sec execute 183 sec latency 71.528 ms 2024-05-09T07:32:32.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 116633 20.29 MB/sec execute 184 sec latency 185.253 ms 2024-05-09T07:32:33.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 117017 20.33 MB/sec execute 185 sec latency 99.337 ms 2024-05-09T07:32:34.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 117307 20.27 MB/sec execute 186 sec latency 87.270 ms 2024-05-09T07:32:35.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 117529 20.20 MB/sec execute 187 sec latency 128.901 ms 2024-05-09T07:32:36.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 117858 20.16 MB/sec execute 188 sec latency 117.297 ms 2024-05-09T07:32:37.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 118333 20.19 MB/sec execute 189 sec latency 108.788 ms 2024-05-09T07:32:38.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 118902 20.21 MB/sec execute 190 sec latency 84.741 ms 2024-05-09T07:32:39.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 119488 20.28 MB/sec execute 191 sec latency 87.721 ms 2024-05-09T07:32:40.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 120129 20.43 MB/sec execute 192 sec latency 50.485 ms 2024-05-09T07:32:41.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 120536 20.47 MB/sec execute 193 sec latency 68.608 ms 2024-05-09T07:32:42.389 INFO:tasks.workunit.client.1.smithi188.stdout: 10 120827 20.42 MB/sec execute 194 sec latency 84.343 ms 2024-05-09T07:32:43.391 INFO:tasks.workunit.client.1.smithi188.stdout: 10 121093 20.36 MB/sec execute 195 sec latency 90.088 ms 2024-05-09T07:32:44.391 INFO:tasks.workunit.client.1.smithi188.stdout: 10 121364 20.31 MB/sec execute 196 sec latency 174.499 ms 2024-05-09T07:32:45.392 INFO:tasks.workunit.client.1.smithi188.stdout: 10 121784 20.34 MB/sec execute 197 sec latency 78.130 ms 2024-05-09T07:32:46.392 INFO:tasks.workunit.client.1.smithi188.stdout: 10 122342 20.34 MB/sec execute 198 sec latency 83.287 ms 2024-05-09T07:32:47.391 INFO:tasks.workunit.client.1.smithi188.stdout: 10 122779 20.33 MB/sec execute 199 sec latency 97.984 ms 2024-05-09T07:32:48.392 INFO:tasks.workunit.client.1.smithi188.stdout: 10 122779 20.23 MB/sec execute 200 sec latency 1096.889 ms 2024-05-09T07:32:49.392 INFO:tasks.workunit.client.1.smithi188.stdout: 10 122801 20.14 MB/sec execute 201 sec latency 2011.609 ms 2024-05-09T07:32:50.392 INFO:tasks.workunit.client.1.smithi188.stdout: 10 123076 20.14 MB/sec execute 202 sec latency 1169.254 ms 2024-05-09T07:32:51.394 INFO:tasks.workunit.client.1.smithi188.stdout: 10 123302 20.11 MB/sec execute 203 sec latency 169.395 ms 2024-05-09T07:32:52.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 123579 20.10 MB/sec execute 204 sec latency 185.148 ms 2024-05-09T07:32:53.394 INFO:tasks.workunit.client.1.smithi188.stdout: 10 123757 20.09 MB/sec execute 205 sec latency 212.017 ms 2024-05-09T07:32:54.394 INFO:tasks.workunit.client.1.smithi188.stdout: 10 124013 20.08 MB/sec execute 206 sec latency 187.463 ms 2024-05-09T07:32:55.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 124290 20.04 MB/sec execute 207 sec latency 185.418 ms 2024-05-09T07:32:56.394 INFO:tasks.workunit.client.1.smithi188.stdout: 10 124517 19.99 MB/sec execute 208 sec latency 170.457 ms 2024-05-09T07:32:57.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 124749 19.94 MB/sec execute 209 sec latency 151.539 ms 2024-05-09T07:32:58.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 125089 19.92 MB/sec execute 210 sec latency 129.790 ms 2024-05-09T07:32:59.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 125559 19.93 MB/sec execute 211 sec latency 124.335 ms 2024-05-09T07:33:00.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 126123 19.96 MB/sec execute 212 sec latency 93.501 ms 2024-05-09T07:33:01.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 126732 20.06 MB/sec execute 213 sec latency 41.832 ms 2024-05-09T07:33:02.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 127201 20.10 MB/sec execute 214 sec latency 95.889 ms 2024-05-09T07:33:03.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 127612 20.14 MB/sec execute 215 sec latency 103.163 ms 2024-05-09T07:33:04.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 127947 20.12 MB/sec execute 216 sec latency 72.647 ms 2024-05-09T07:33:05.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 128146 20.05 MB/sec execute 217 sec latency 159.620 ms 2024-05-09T07:33:06.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 128455 20.02 MB/sec execute 218 sec latency 111.381 ms 2024-05-09T07:33:07.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 128864 20.03 MB/sec execute 219 sec latency 50.183 ms 2024-05-09T07:33:08.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 129273 20.00 MB/sec execute 220 sec latency 122.820 ms 2024-05-09T07:33:09.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 129844 20.05 MB/sec execute 221 sec latency 43.286 ms 2024-05-09T07:33:10.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 130393 20.12 MB/sec execute 222 sec latency 96.548 ms 2024-05-09T07:33:11.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 130871 20.18 MB/sec execute 223 sec latency 56.453 ms 2024-05-09T07:33:12.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131210 20.19 MB/sec execute 224 sec latency 128.795 ms 2024-05-09T07:33:13.395 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131392 20.14 MB/sec execute 225 sec latency 523.775 ms 2024-05-09T07:33:14.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131407 20.05 MB/sec execute 226 sec latency 1301.593 ms 2024-05-09T07:33:15.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131443 19.98 MB/sec execute 227 sec latency 1020.709 ms 2024-05-09T07:33:16.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131606 19.92 MB/sec execute 228 sec latency 302.634 ms 2024-05-09T07:33:17.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131783 19.87 MB/sec execute 229 sec latency 178.018 ms 2024-05-09T07:33:18.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 131942 19.80 MB/sec execute 230 sec latency 146.974 ms 2024-05-09T07:33:19.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 132134 19.76 MB/sec execute 231 sec latency 281.634 ms 2024-05-09T07:33:20.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 132552 19.75 MB/sec execute 232 sec latency 149.816 ms 2024-05-09T07:33:21.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 133063 19.76 MB/sec execute 233 sec latency 115.434 ms 2024-05-09T07:33:22.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 133671 19.80 MB/sec execute 234 sec latency 44.244 ms 2024-05-09T07:33:23.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 134164 19.86 MB/sec execute 235 sec latency 97.677 ms 2024-05-09T07:33:24.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 134620 19.94 MB/sec execute 236 sec latency 114.129 ms 2024-05-09T07:33:25.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 135013 19.95 MB/sec execute 237 sec latency 91.370 ms 2024-05-09T07:33:26.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 135202 19.89 MB/sec execute 238 sec latency 106.576 ms 2024-05-09T07:33:27.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 135473 19.86 MB/sec execute 239 sec latency 132.821 ms 2024-05-09T07:33:28.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 135837 19.83 MB/sec execute 240 sec latency 114.486 ms 2024-05-09T07:33:29.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 136375 19.85 MB/sec execute 241 sec latency 83.345 ms 2024-05-09T07:33:30.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 137105 19.91 MB/sec execute 242 sec latency 53.633 ms 2024-05-09T07:33:31.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 137644 19.98 MB/sec execute 243 sec latency 109.996 ms 2024-05-09T07:33:32.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138155 20.07 MB/sec execute 244 sec latency 82.094 ms 2024-05-09T07:33:33.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138459 20.06 MB/sec execute 245 sec latency 109.816 ms 2024-05-09T07:33:34.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138477 19.99 MB/sec execute 246 sec latency 857.310 ms 2024-05-09T07:33:35.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138496 19.91 MB/sec execute 247 sec latency 1010.354 ms 2024-05-09T07:33:36.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138631 19.85 MB/sec execute 248 sec latency 460.519 ms 2024-05-09T07:33:37.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138751 19.78 MB/sec execute 249 sec latency 92.969 ms 2024-05-09T07:33:38.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 138995 19.74 MB/sec execute 250 sec latency 122.384 ms 2024-05-09T07:33:39.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 139263 19.70 MB/sec execute 251 sec latency 198.174 ms 2024-05-09T07:33:40.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 139719 19.72 MB/sec execute 252 sec latency 109.856 ms 2024-05-09T07:33:41.396 INFO:tasks.workunit.client.1.smithi188.stdout: 10 140437 19.77 MB/sec execute 253 sec latency 47.449 ms 2024-05-09T07:33:42.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 140925 19.79 MB/sec execute 254 sec latency 95.229 ms 2024-05-09T07:33:43.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 141428 19.88 MB/sec execute 255 sec latency 108.039 ms 2024-05-09T07:33:44.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 141842 19.93 MB/sec execute 256 sec latency 85.253 ms 2024-05-09T07:33:45.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 142116 19.89 MB/sec execute 257 sec latency 96.098 ms 2024-05-09T07:33:46.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 142366 19.83 MB/sec execute 258 sec latency 114.368 ms 2024-05-09T07:33:47.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 142640 19.80 MB/sec execute 259 sec latency 139.102 ms 2024-05-09T07:33:48.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 143061 19.80 MB/sec execute 260 sec latency 119.686 ms 2024-05-09T07:33:49.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 143716 19.85 MB/sec execute 261 sec latency 46.875 ms 2024-05-09T07:33:50.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 144329 19.88 MB/sec execute 262 sec latency 89.560 ms 2024-05-09T07:33:51.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 144882 19.96 MB/sec execute 263 sec latency 104.641 ms 2024-05-09T07:33:52.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 145294 20.01 MB/sec execute 264 sec latency 110.914 ms 2024-05-09T07:33:53.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 145577 20.00 MB/sec execute 265 sec latency 125.038 ms 2024-05-09T07:33:54.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 145807 19.95 MB/sec execute 266 sec latency 148.570 ms 2024-05-09T07:33:55.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 146057 19.91 MB/sec execute 267 sec latency 145.245 ms 2024-05-09T07:33:56.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 146465 19.89 MB/sec execute 268 sec latency 88.678 ms 2024-05-09T07:33:57.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 147024 19.93 MB/sec execute 269 sec latency 52.272 ms 2024-05-09T07:33:58.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 147734 19.99 MB/sec execute 270 sec latency 34.268 ms 2024-05-09T07:33:59.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 148285 20.04 MB/sec execute 271 sec latency 106.634 ms 2024-05-09T07:34:00.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 148694 20.09 MB/sec execute 272 sec latency 114.060 ms 2024-05-09T07:34:01.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 149025 20.09 MB/sec execute 273 sec latency 131.740 ms 2024-05-09T07:34:02.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 149236 20.05 MB/sec execute 274 sec latency 122.909 ms 2024-05-09T07:34:03.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 149509 20.00 MB/sec execute 275 sec latency 117.835 ms 2024-05-09T07:34:04.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 149772 19.97 MB/sec execute 276 sec latency 167.879 ms 2024-05-09T07:34:05.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 150205 19.97 MB/sec execute 277 sec latency 107.129 ms 2024-05-09T07:34:06.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 150794 20.01 MB/sec execute 278 sec latency 64.354 ms 2024-05-09T07:34:07.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 151389 20.05 MB/sec execute 279 sec latency 68.650 ms 2024-05-09T07:34:08.397 INFO:tasks.workunit.client.1.smithi188.stdout: 10 151846 20.09 MB/sec execute 280 sec latency 123.910 ms 2024-05-09T07:34:09.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152188 20.12 MB/sec execute 281 sec latency 215.562 ms 2024-05-09T07:34:10.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152208 20.05 MB/sec execute 282 sec latency 1175.009 ms 2024-05-09T07:34:11.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152208 19.98 MB/sec execute 283 sec latency 2175.048 ms 2024-05-09T07:34:12.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152214 19.91 MB/sec execute 284 sec latency 3158.060 ms 2024-05-09T07:34:13.401 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152296 19.86 MB/sec execute 285 sec latency 3466.827 ms 2024-05-09T07:34:14.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152441 19.82 MB/sec execute 286 sec latency 173.092 ms 2024-05-09T07:34:15.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152601 19.78 MB/sec execute 287 sec latency 123.797 ms 2024-05-09T07:34:16.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152735 19.74 MB/sec execute 288 sec latency 112.877 ms 2024-05-09T07:34:17.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 152865 19.68 MB/sec execute 289 sec latency 135.299 ms 2024-05-09T07:34:18.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 153030 19.62 MB/sec execute 290 sec latency 229.543 ms 2024-05-09T07:34:19.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 153209 19.58 MB/sec execute 291 sec latency 228.266 ms 2024-05-09T07:34:20.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 153580 19.57 MB/sec execute 292 sec latency 133.599 ms 2024-05-09T07:34:21.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 154124 19.59 MB/sec execute 293 sec latency 104.775 ms 2024-05-09T07:34:22.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 154738 19.64 MB/sec execute 294 sec latency 94.155 ms 2024-05-09T07:34:23.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 155262 19.68 MB/sec execute 295 sec latency 114.128 ms 2024-05-09T07:34:24.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 155705 19.74 MB/sec execute 296 sec latency 158.325 ms 2024-05-09T07:34:25.398 INFO:tasks.workunit.client.1.smithi188.stdout: 10 156059 19.75 MB/sec execute 297 sec latency 95.348 ms 2024-05-09T07:34:26.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 156335 19.72 MB/sec execute 298 sec latency 105.497 ms 2024-05-09T07:34:27.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 156619 19.67 MB/sec execute 299 sec latency 82.352 ms 2024-05-09T07:34:28.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 157039 19.68 MB/sec execute 300 sec latency 71.534 ms 2024-05-09T07:34:29.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 157698 19.71 MB/sec execute 301 sec latency 74.924 ms 2024-05-09T07:34:30.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 157951 19.69 MB/sec execute 302 sec latency 711.923 ms 2024-05-09T07:34:31.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 158052 19.64 MB/sec execute 303 sec latency 1442.174 ms 2024-05-09T07:34:32.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 158332 19.63 MB/sec execute 304 sec latency 109.683 ms 2024-05-09T07:34:33.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 158856 19.68 MB/sec execute 305 sec latency 101.641 ms 2024-05-09T07:34:34.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 159272 19.72 MB/sec execute 306 sec latency 238.005 ms 2024-05-09T07:34:35.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 159613 19.73 MB/sec execute 307 sec latency 288.771 ms 2024-05-09T07:34:36.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 159896 19.70 MB/sec execute 308 sec latency 76.164 ms 2024-05-09T07:34:37.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 160119 19.65 MB/sec execute 309 sec latency 156.512 ms 2024-05-09T07:34:38.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 160478 19.64 MB/sec execute 310 sec latency 142.333 ms 2024-05-09T07:34:39.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 161156 19.68 MB/sec execute 311 sec latency 105.283 ms 2024-05-09T07:34:40.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 161814 19.73 MB/sec execute 312 sec latency 59.271 ms 2024-05-09T07:34:41.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 162435 19.80 MB/sec execute 313 sec latency 119.391 ms 2024-05-09T07:34:42.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 162900 19.85 MB/sec execute 314 sec latency 72.635 ms 2024-05-09T07:34:43.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 163234 19.85 MB/sec execute 315 sec latency 79.581 ms 2024-05-09T07:34:44.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 163493 19.81 MB/sec execute 316 sec latency 80.840 ms 2024-05-09T07:34:45.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 163729 19.77 MB/sec execute 317 sec latency 356.940 ms 2024-05-09T07:34:46.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 163749 19.70 MB/sec execute 318 sec latency 1243.529 ms 2024-05-09T07:34:47.400 INFO:tasks.workunit.client.1.smithi188.stdout: 10 163903 19.67 MB/sec execute 319 sec latency 432.140 ms 2024-05-09T07:34:48.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 164135 19.64 MB/sec execute 320 sec latency 117.249 ms 2024-05-09T07:34:49.399 INFO:tasks.workunit.client.1.smithi188.stdout: 10 164806 19.67 MB/sec execute 321 sec latency 95.920 ms 2024-05-09T07:34:50.400 INFO:tasks.workunit.client.1.smithi188.stdout: 10 165541 19.74 MB/sec execute 322 sec latency 60.074 ms 2024-05-09T07:34:51.400 INFO:tasks.workunit.client.1.smithi188.stdout: 10 166073 19.79 MB/sec execute 323 sec latency 95.965 ms 2024-05-09T07:34:52.400 INFO:tasks.workunit.client.1.smithi188.stdout: 10 166540 19.86 MB/sec execute 324 sec latency 121.702 ms 2024-05-09T07:34:53.400 INFO:tasks.workunit.client.1.smithi188.stdout: 10 166805 19.83 MB/sec execute 325 sec latency 106.247 ms 2024-05-09T07:34:54.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 166983 19.79 MB/sec execute 326 sec latency 168.036 ms 2024-05-09T07:34:55.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 167268 19.74 MB/sec execute 327 sec latency 164.701 ms 2024-05-09T07:34:55.818 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:34:55.821 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:34:55.857 INFO:tasks.ceph.mds.d.smithi188.stderr:2024-05-09T07:34:55.847+0000 7fa6c2e0d640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42929) UID: 0 2024-05-09T07:34:55.857 INFO:tasks.ceph.mds.b.smithi188.stderr:2024-05-09T07:34:55.847+0000 7f44cf7fa640 -1 received signal: Hangup from killall ceph-mds -1 (PID: 42929) UID: 0 2024-05-09T07:34:56.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 167607 19.73 MB/sec execute 328 sec latency 134.574 ms 2024-05-09T07:34:57.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 168197 19.75 MB/sec execute 329 sec latency 109.968 ms 2024-05-09T07:34:58.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 168837 19.80 MB/sec execute 330 sec latency 74.833 ms 2024-05-09T07:34:59.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 169402 19.85 MB/sec execute 331 sec latency 131.425 ms 2024-05-09T07:35:00.402 INFO:tasks.workunit.client.1.smithi188.stdout: 10 169887 19.91 MB/sec execute 332 sec latency 54.827 ms 2024-05-09T07:35:01.406 INFO:tasks.workunit.client.1.smithi188.stdout: 10 170167 19.91 MB/sec execute 333 sec latency 125.490 ms 2024-05-09T07:35:02.406 INFO:tasks.workunit.client.1.smithi188.stdout: 10 170376 19.87 MB/sec execute 334 sec latency 118.532 ms 2024-05-09T07:35:03.406 INFO:tasks.workunit.client.1.smithi188.stdout: 10 170590 19.83 MB/sec execute 335 sec latency 159.397 ms 2024-05-09T07:35:04.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 170832 19.79 MB/sec execute 336 sec latency 113.049 ms 2024-05-09T07:35:05.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 171173 19.77 MB/sec execute 337 sec latency 131.320 ms 2024-05-09T07:35:06.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 171710 19.79 MB/sec execute 338 sec latency 110.420 ms 2024-05-09T07:35:07.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 172389 19.84 MB/sec execute 339 sec latency 40.300 ms 2024-05-09T07:35:08.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 172991 19.90 MB/sec execute 340 sec latency 40.982 ms 2024-05-09T07:35:09.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173386 19.93 MB/sec execute 341 sec latency 139.990 ms 2024-05-09T07:35:10.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173720 19.94 MB/sec execute 342 sec latency 115.538 ms 2024-05-09T07:35:11.407 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173764 19.89 MB/sec execute 343 sec latency 789.997 ms 2024-05-09T07:35:12.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173771 19.83 MB/sec execute 344 sec latency 1767.122 ms 2024-05-09T07:35:13.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173782 19.77 MB/sec execute 345 sec latency 2274.053 ms 2024-05-09T07:35:14.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173846 19.72 MB/sec execute 346 sec latency 1077.977 ms 2024-05-09T07:35:15.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 173979 19.68 MB/sec execute 347 sec latency 259.482 ms 2024-05-09T07:35:16.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 174099 19.63 MB/sec execute 348 sec latency 181.490 ms 2024-05-09T07:35:17.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 174319 19.60 MB/sec execute 349 sec latency 193.647 ms 2024-05-09T07:35:18.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 174639 19.58 MB/sec execute 350 sec latency 109.767 ms 2024-05-09T07:35:19.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 175207 19.58 MB/sec execute 351 sec latency 142.858 ms 2024-05-09T07:35:20.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 175883 19.64 MB/sec execute 352 sec latency 101.732 ms 2024-05-09T07:35:21.411 INFO:tasks.workunit.client.1.smithi188.stdout: 10 176553 19.71 MB/sec execute 353 sec latency 61.672 ms 2024-05-09T07:35:22.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 176981 19.75 MB/sec execute 354 sec latency 85.072 ms 2024-05-09T07:35:23.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 177313 19.75 MB/sec execute 355 sec latency 101.683 ms 2024-05-09T07:35:24.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 177511 19.71 MB/sec execute 356 sec latency 86.534 ms 2024-05-09T07:35:25.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 177771 19.68 MB/sec execute 357 sec latency 104.481 ms 2024-05-09T07:35:26.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 178119 19.66 MB/sec execute 358 sec latency 87.496 ms 2024-05-09T07:35:27.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 178735 19.67 MB/sec execute 359 sec latency 84.273 ms 2024-05-09T07:35:28.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 179416 19.74 MB/sec execute 360 sec latency 36.030 ms 2024-05-09T07:35:29.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 180065 19.79 MB/sec execute 361 sec latency 43.786 ms 2024-05-09T07:35:30.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 180505 19.84 MB/sec execute 362 sec latency 117.468 ms 2024-05-09T07:35:31.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 180815 19.84 MB/sec execute 363 sec latency 109.254 ms 2024-05-09T07:35:32.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 180972 19.80 MB/sec execute 364 sec latency 265.109 ms 2024-05-09T07:35:33.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 181010 19.75 MB/sec execute 365 sec latency 1265.145 ms 2024-05-09T07:35:34.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 181041 19.70 MB/sec execute 366 sec latency 2088.443 ms 2024-05-09T07:35:35.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 181159 19.65 MB/sec execute 367 sec latency 190.835 ms 2024-05-09T07:35:36.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 181312 19.61 MB/sec execute 368 sec latency 145.128 ms 2024-05-09T07:35:37.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 181574 19.59 MB/sec execute 369 sec latency 130.671 ms 2024-05-09T07:35:38.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 182034 19.58 MB/sec execute 370 sec latency 125.983 ms 2024-05-09T07:35:39.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 182601 19.61 MB/sec execute 371 sec latency 119.040 ms 2024-05-09T07:35:40.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 183394 19.67 MB/sec execute 372 sec latency 43.702 ms 2024-05-09T07:35:41.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 183871 19.73 MB/sec execute 373 sec latency 98.307 ms 2024-05-09T07:35:42.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 184264 19.74 MB/sec execute 374 sec latency 86.094 ms 2024-05-09T07:35:43.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 184494 19.73 MB/sec execute 375 sec latency 126.398 ms 2024-05-09T07:35:44.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 184733 19.69 MB/sec execute 376 sec latency 85.772 ms 2024-05-09T07:35:45.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 185083 19.68 MB/sec execute 377 sec latency 80.594 ms 2024-05-09T07:35:46.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 185458 19.66 MB/sec execute 378 sec latency 140.520 ms 2024-05-09T07:35:47.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 186089 19.69 MB/sec execute 379 sec latency 88.388 ms 2024-05-09T07:35:48.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 186867 19.75 MB/sec execute 380 sec latency 36.543 ms 2024-05-09T07:35:49.412 INFO:tasks.workunit.client.1.smithi188.stdout: 10 187286 19.79 MB/sec execute 381 sec latency 91.163 ms 2024-05-09T07:35:50.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 187733 19.82 MB/sec execute 382 sec latency 119.909 ms 2024-05-09T07:35:51.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 188003 19.81 MB/sec execute 383 sec latency 112.001 ms 2024-05-09T07:35:52.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 188227 19.78 MB/sec execute 384 sec latency 127.280 ms 2024-05-09T07:35:53.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 188517 19.76 MB/sec execute 385 sec latency 118.402 ms 2024-05-09T07:35:54.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 188866 19.73 MB/sec execute 386 sec latency 97.710 ms 2024-05-09T07:35:55.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 189458 19.75 MB/sec execute 387 sec latency 49.019 ms 2024-05-09T07:35:56.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 190195 19.81 MB/sec execute 388 sec latency 62.286 ms 2024-05-09T07:35:57.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 190782 19.85 MB/sec execute 389 sec latency 87.956 ms 2024-05-09T07:35:58.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 191231 19.89 MB/sec execute 390 sec latency 143.412 ms 2024-05-09T07:35:59.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 191526 19.89 MB/sec execute 391 sec latency 170.953 ms 2024-05-09T07:36:00.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 191733 19.86 MB/sec execute 392 sec latency 129.302 ms 2024-05-09T07:36:01.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 192025 19.84 MB/sec execute 393 sec latency 117.487 ms 2024-05-09T07:36:02.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 192357 19.81 MB/sec execute 394 sec latency 129.985 ms 2024-05-09T07:36:03.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 192797 19.81 MB/sec execute 395 sec latency 96.743 ms 2024-05-09T07:36:04.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 193441 19.86 MB/sec execute 396 sec latency 59.829 ms 2024-05-09T07:36:05.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 194092 19.89 MB/sec execute 397 sec latency 103.817 ms 2024-05-09T07:36:06.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 194584 19.95 MB/sec execute 398 sec latency 49.931 ms 2024-05-09T07:36:07.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 194905 19.94 MB/sec execute 399 sec latency 160.382 ms 2024-05-09T07:36:08.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195182 19.93 MB/sec execute 400 sec latency 102.646 ms 2024-05-09T07:36:09.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195207 19.89 MB/sec execute 401 sec latency 927.647 ms 2024-05-09T07:36:10.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195208 19.84 MB/sec execute 402 sec latency 1925.094 ms 2024-05-09T07:36:11.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195213 19.79 MB/sec execute 403 sec latency 2859.946 ms 2024-05-09T07:36:12.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195221 19.74 MB/sec execute 404 sec latency 2980.954 ms 2024-05-09T07:36:13.413 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195350 19.70 MB/sec execute 405 sec latency 624.041 ms 2024-05-09T07:36:14.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195453 19.66 MB/sec execute 406 sec latency 191.543 ms 2024-05-09T07:36:15.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195546 19.62 MB/sec execute 407 sec latency 224.235 ms 2024-05-09T07:36:16.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195700 19.59 MB/sec execute 408 sec latency 318.193 ms 2024-05-09T07:36:17.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195842 19.55 MB/sec execute 409 sec latency 193.050 ms 2024-05-09T07:36:18.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 195964 19.51 MB/sec execute 410 sec latency 242.670 ms 2024-05-09T07:36:19.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 196407 19.52 MB/sec execute 411 sec latency 124.186 ms 2024-05-09T07:36:20.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 197055 19.56 MB/sec execute 412 sec latency 89.043 ms 2024-05-09T07:36:21.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 197671 19.59 MB/sec execute 413 sec latency 65.613 ms 2024-05-09T07:36:22.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 198152 19.64 MB/sec execute 414 sec latency 114.624 ms 2024-05-09T07:36:23.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 198524 19.66 MB/sec execute 415 sec latency 91.238 ms 2024-05-09T07:36:24.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 198741 19.63 MB/sec execute 416 sec latency 158.421 ms 2024-05-09T07:36:25.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 198986 19.60 MB/sec execute 417 sec latency 94.542 ms 2024-05-09T07:36:26.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 199271 19.58 MB/sec execute 418 sec latency 146.066 ms 2024-05-09T07:36:27.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 199709 19.57 MB/sec execute 419 sec latency 94.767 ms 2024-05-09T07:36:28.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 200405 19.62 MB/sec execute 420 sec latency 57.549 ms 2024-05-09T07:36:29.414 INFO:tasks.workunit.client.1.smithi188.stdout: 10 200877 19.61 MB/sec execute 421 sec latency 432.951 ms 2024-05-09T07:36:30.415 INFO:tasks.workunit.client.1.smithi188.stdout: 10 200893 19.57 MB/sec execute 422 sec latency 1433.735 ms 2024-05-09T07:36:31.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 201092 19.56 MB/sec execute 423 sec latency 1454.930 ms 2024-05-09T07:36:32.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 201341 19.56 MB/sec execute 424 sec latency 133.768 ms 2024-05-09T07:36:33.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 201763 19.59 MB/sec execute 425 sec latency 92.009 ms 2024-05-09T07:36:34.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 202104 19.60 MB/sec execute 426 sec latency 169.218 ms 2024-05-09T07:36:35.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 202343 19.58 MB/sec execute 427 sec latency 171.351 ms 2024-05-09T07:36:36.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 202618 19.55 MB/sec execute 428 sec latency 129.460 ms 2024-05-09T07:36:37.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 202982 19.53 MB/sec execute 429 sec latency 111.860 ms 2024-05-09T07:36:38.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 203496 19.55 MB/sec execute 430 sec latency 113.942 ms 2024-05-09T07:36:39.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 204198 19.59 MB/sec execute 431 sec latency 50.322 ms 2024-05-09T07:36:40.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 204826 19.63 MB/sec execute 432 sec latency 75.228 ms 2024-05-09T07:36:41.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 205270 19.67 MB/sec execute 433 sec latency 81.720 ms 2024-05-09T07:36:42.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 205618 19.67 MB/sec execute 434 sec latency 122.595 ms 2024-05-09T07:36:43.422 INFO:tasks.workunit.client.1.smithi188.stdout: 10 205882 19.65 MB/sec execute 435 sec latency 161.194 ms 2024-05-09T07:36:44.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 206091 19.62 MB/sec execute 436 sec latency 222.637 ms 2024-05-09T07:36:45.418 INFO:tasks.workunit.client.1.smithi188.stdout: 10 206107 19.58 MB/sec execute 437 sec latency 1101.049 ms 2024-05-09T07:36:46.417 INFO:tasks.workunit.client.1.smithi188.stdout: 10 206140 19.54 MB/sec execute 438 sec latency 1517.183 ms 2024-05-09T07:36:47.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 206370 19.51 MB/sec execute 439 sec latency 136.813 ms 2024-05-09T07:36:48.418 INFO:tasks.workunit.client.1.smithi188.stdout: 10 206617 19.49 MB/sec execute 440 sec latency 96.316 ms 2024-05-09T07:36:49.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 207037 19.49 MB/sec execute 441 sec latency 189.022 ms 2024-05-09T07:36:50.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 207714 19.53 MB/sec execute 442 sec latency 62.443 ms 2024-05-09T07:36:51.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 208326 19.57 MB/sec execute 443 sec latency 99.065 ms 2024-05-09T07:36:52.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 208774 19.61 MB/sec execute 444 sec latency 84.317 ms 2024-05-09T07:36:53.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 209091 19.60 MB/sec execute 445 sec latency 95.885 ms 2024-05-09T07:36:54.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 209347 19.59 MB/sec execute 446 sec latency 129.372 ms 2024-05-09T07:36:55.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 209569 19.56 MB/sec execute 447 sec latency 140.308 ms 2024-05-09T07:36:56.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 209858 19.54 MB/sec execute 448 sec latency 102.208 ms 2024-05-09T07:36:57.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 210309 19.53 MB/sec execute 449 sec latency 91.986 ms 2024-05-09T07:36:58.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 210905 19.56 MB/sec execute 450 sec latency 72.546 ms 2024-05-09T07:36:59.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 211512 19.59 MB/sec execute 451 sec latency 117.972 ms 2024-05-09T07:37:00.422 INFO:tasks.workunit.client.1.smithi188.stdout: 10 212150 19.64 MB/sec execute 452 sec latency 40.486 ms 2024-05-09T07:37:01.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 212558 19.67 MB/sec execute 453 sec latency 106.968 ms 2024-05-09T07:37:02.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 212831 19.66 MB/sec execute 454 sec latency 145.219 ms 2024-05-09T07:37:03.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 213084 19.64 MB/sec execute 455 sec latency 92.675 ms 2024-05-09T07:37:04.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 213339 19.61 MB/sec execute 456 sec latency 212.765 ms 2024-05-09T07:37:05.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 213714 19.60 MB/sec execute 457 sec latency 159.179 ms 2024-05-09T07:37:06.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 214198 19.61 MB/sec execute 458 sec latency 92.869 ms 2024-05-09T07:37:07.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 214885 19.64 MB/sec execute 459 sec latency 46.047 ms 2024-05-09T07:37:08.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 215514 19.68 MB/sec execute 460 sec latency 106.860 ms 2024-05-09T07:37:09.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 215947 19.72 MB/sec execute 461 sec latency 111.040 ms 2024-05-09T07:37:10.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216130 19.70 MB/sec execute 462 sec latency 460.469 ms 2024-05-09T07:37:11.419 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216160 19.66 MB/sec execute 463 sec latency 1389.424 ms 2024-05-09T07:37:12.421 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216164 19.62 MB/sec execute 464 sec latency 1804.479 ms 2024-05-09T07:37:13.421 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216209 19.58 MB/sec execute 465 sec latency 770.894 ms 2024-05-09T07:37:14.421 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216355 19.56 MB/sec execute 466 sec latency 195.461 ms 2024-05-09T07:37:15.421 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216445 19.53 MB/sec execute 467 sec latency 272.250 ms 2024-05-09T07:37:16.421 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216544 19.49 MB/sec execute 468 sec latency 194.490 ms 2024-05-09T07:37:17.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 216806 19.47 MB/sec execute 469 sec latency 196.516 ms 2024-05-09T07:37:18.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 217007 19.44 MB/sec execute 470 sec latency 170.103 ms 2024-05-09T07:37:19.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 217423 19.43 MB/sec execute 471 sec latency 159.484 ms 2024-05-09T07:37:20.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 217946 19.45 MB/sec execute 472 sec latency 104.184 ms 2024-05-09T07:37:21.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 218716 19.50 MB/sec execute 473 sec latency 39.439 ms 2024-05-09T07:37:22.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 219307 19.54 MB/sec execute 474 sec latency 53.363 ms 2024-05-09T07:37:23.423 INFO:tasks.workunit.client.1.smithi188.stdout: 10 219654 19.56 MB/sec execute 475 sec latency 85.995 ms 2024-05-09T07:37:24.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 219931 19.56 MB/sec execute 476 sec latency 119.748 ms 2024-05-09T07:37:25.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 220132 19.53 MB/sec execute 477 sec latency 133.624 ms 2024-05-09T07:37:26.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 220355 19.50 MB/sec execute 478 sec latency 137.800 ms 2024-05-09T07:37:27.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 220655 19.49 MB/sec execute 479 sec latency 181.403 ms 2024-05-09T07:37:28.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 221110 19.48 MB/sec execute 480 sec latency 150.618 ms 2024-05-09T07:37:29.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 221710 19.51 MB/sec execute 481 sec latency 79.604 ms 2024-05-09T07:37:30.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 222399 19.55 MB/sec execute 482 sec latency 40.671 ms 2024-05-09T07:37:31.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 222438 19.52 MB/sec execute 483 sec latency 1007.666 ms 2024-05-09T07:37:32.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 222466 19.48 MB/sec execute 484 sec latency 1369.792 ms 2024-05-09T07:37:33.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 222715 19.47 MB/sec execute 485 sec latency 532.909 ms 2024-05-09T07:37:34.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 222847 19.46 MB/sec execute 486 sec latency 323.039 ms 2024-05-09T07:37:35.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 223297 19.48 MB/sec execute 487 sec latency 124.140 ms 2024-05-09T07:37:36.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 223566 19.47 MB/sec execute 488 sec latency 84.908 ms 2024-05-09T07:37:37.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 223767 19.44 MB/sec execute 489 sec latency 116.207 ms 2024-05-09T07:37:38.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 224066 19.43 MB/sec execute 490 sec latency 125.392 ms 2024-05-09T07:37:39.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 224501 19.41 MB/sec execute 491 sec latency 93.587 ms 2024-05-09T07:37:40.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 225150 19.45 MB/sec execute 492 sec latency 77.613 ms 2024-05-09T07:37:41.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 225921 19.50 MB/sec execute 493 sec latency 33.505 ms 2024-05-09T07:37:42.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 226453 19.54 MB/sec execute 494 sec latency 86.431 ms 2024-05-09T07:37:43.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 226890 19.56 MB/sec execute 495 sec latency 102.823 ms 2024-05-09T07:37:44.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 227157 19.54 MB/sec execute 496 sec latency 89.801 ms 2024-05-09T07:37:45.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 227431 19.52 MB/sec execute 497 sec latency 101.597 ms 2024-05-09T07:37:46.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 227556 19.49 MB/sec execute 498 sec latency 666.798 ms 2024-05-09T07:37:47.424 INFO:tasks.workunit.client.1.smithi188.stdout: 10 227633 19.46 MB/sec execute 499 sec latency 1231.874 ms 2024-05-09T07:37:48.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 227810 19.44 MB/sec execute 500 sec latency 167.660 ms 2024-05-09T07:37:49.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 228108 19.42 MB/sec execute 501 sec latency 171.626 ms 2024-05-09T07:37:50.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 228723 19.44 MB/sec execute 502 sec latency 115.970 ms 2024-05-09T07:37:51.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 229452 19.48 MB/sec execute 503 sec latency 35.331 ms 2024-05-09T07:37:52.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 230032 19.53 MB/sec execute 504 sec latency 39.801 ms 2024-05-09T07:37:53.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 230399 19.55 MB/sec execute 505 sec latency 150.622 ms 2024-05-09T07:37:54.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 230658 19.54 MB/sec execute 506 sec latency 77.651 ms 2024-05-09T07:37:55.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 230869 19.51 MB/sec execute 507 sec latency 118.927 ms 2024-05-09T07:37:56.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 231117 19.49 MB/sec execute 508 sec latency 128.519 ms 2024-05-09T07:37:57.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 231631 19.49 MB/sec execute 509 sec latency 103.071 ms 2024-05-09T07:37:58.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 232372 19.52 MB/sec execute 510 sec latency 46.627 ms 2024-05-09T07:37:59.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 233113 19.56 MB/sec execute 511 sec latency 36.317 ms 2024-05-09T07:38:00.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 233678 19.62 MB/sec execute 512 sec latency 135.913 ms 2024-05-09T07:38:01.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 234072 19.64 MB/sec execute 513 sec latency 77.482 ms 2024-05-09T07:38:02.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 234272 19.61 MB/sec execute 514 sec latency 135.673 ms 2024-05-09T07:38:03.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 234464 19.58 MB/sec execute 515 sec latency 148.318 ms 2024-05-09T07:38:04.294 DEBUG:teuthology.orchestra.run.smithi160:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:38:04.297 DEBUG:teuthology.orchestra.run.smithi188:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-05-09T07:38:04.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 234721 19.56 MB/sec execute 516 sec latency 104.410 ms 2024-05-09T07:38:04.639 INFO:tasks.ceph.osd.0.smithi160.stderr:2024-05-09T07:38:04.629+0000 7f3bb0795640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48042) UID: 0 2024-05-09T07:38:04.640 INFO:tasks.ceph.osd.1.smithi160.stderr:2024-05-09T07:38:04.630+0000 7f0cbe8ef640 -1 received signal: Hangup from (PID: 48042) UID: 0 2024-05-09T07:38:04.640 INFO:tasks.ceph.osd.2.smithi160.stderr:2024-05-09T07:38:04.629+0000 7f2a6fe99640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48042) UID: 0 2024-05-09T07:38:04.640 INFO:tasks.ceph.osd.3.smithi160.stderr:2024-05-09T07:38:04.629+0000 7ff79c9a7640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 48042) UID: 0 2024-05-09T07:38:05.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 235221 19.56 MB/sec execute 517 sec latency 110.374 ms 2024-05-09T07:38:06.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 235846 19.58 MB/sec execute 518 sec latency 170.730 ms 2024-05-09T07:38:07.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 235859 19.54 MB/sec execute 519 sec latency 1116.324 ms 2024-05-09T07:38:08.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 235919 19.51 MB/sec execute 520 sec latency 1579.628 ms 2024-05-09T07:38:09.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 236314 19.51 MB/sec execute 521 sec latency 215.377 ms 2024-05-09T07:38:10.425 INFO:tasks.workunit.client.1.smithi188.stdout: 10 236566 19.51 MB/sec execute 522 sec latency 131.055 ms 2024-05-09T07:38:11.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 236907 19.53 MB/sec execute 523 sec latency 125.833 ms 2024-05-09T07:38:12.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 237341 19.55 MB/sec execute 524 sec latency 121.429 ms 2024-05-09T07:38:13.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 237641 19.55 MB/sec execute 525 sec latency 102.818 ms 2024-05-09T07:38:14.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 237836 19.52 MB/sec execute 526 sec latency 88.443 ms 2024-05-09T07:38:15.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 238011 19.49 MB/sec execute 527 sec latency 159.413 ms 2024-05-09T07:38:16.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 238323 19.48 MB/sec execute 528 sec latency 116.382 ms 2024-05-09T07:38:17.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 238818 19.48 MB/sec execute 529 sec latency 138.635 ms 2024-05-09T07:38:18.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 239557 19.51 MB/sec execute 530 sec latency 50.799 ms 2024-05-09T07:38:19.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 240258 19.55 MB/sec execute 531 sec latency 67.029 ms 2024-05-09T07:38:20.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 240784 19.60 MB/sec execute 532 sec latency 84.520 ms 2024-05-09T07:38:21.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 241113 19.61 MB/sec execute 533 sec latency 134.551 ms 2024-05-09T07:38:22.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 241170 19.58 MB/sec execute 534 sec latency 792.429 ms 2024-05-09T07:38:23.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 241215 19.54 MB/sec execute 535 sec latency 1571.834 ms 2024-05-09T07:38:24.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 241352 19.51 MB/sec execute 536 sec latency 140.340 ms 2024-05-09T07:38:25.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 241466 19.48 MB/sec execute 537 sec latency 155.454 ms 2024-05-09T07:38:26.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 241762 19.47 MB/sec execute 538 sec latency 111.295 ms 2024-05-09T07:38:27.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 242079 19.45 MB/sec execute 539 sec latency 119.978 ms 2024-05-09T07:38:28.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 242699 19.47 MB/sec execute 540 sec latency 66.951 ms 2024-05-09T07:38:29.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 243417 19.51 MB/sec execute 541 sec latency 37.253 ms 2024-05-09T07:38:30.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 244016 19.55 MB/sec execute 542 sec latency 135.888 ms 2024-05-09T07:38:31.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 244429 19.58 MB/sec execute 543 sec latency 117.015 ms 2024-05-09T07:38:32.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 244700 19.56 MB/sec execute 544 sec latency 271.369 ms 2024-05-09T07:38:33.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 244959 19.55 MB/sec execute 545 sec latency 130.086 ms 2024-05-09T07:38:34.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 245230 19.52 MB/sec execute 546 sec latency 93.974 ms 2024-05-09T07:38:35.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 245619 19.52 MB/sec execute 547 sec latency 100.218 ms 2024-05-09T07:38:36.426 INFO:tasks.workunit.client.1.smithi188.stdout: 10 246211 19.53 MB/sec execute 548 sec latency 78.206 ms 2024-05-09T07:38:37.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 246906 19.57 MB/sec execute 549 sec latency 55.297 ms 2024-05-09T07:38:38.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 247591 19.62 MB/sec execute 550 sec latency 47.987 ms 2024-05-09T07:38:39.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 247993 19.64 MB/sec execute 551 sec latency 77.683 ms 2024-05-09T07:38:40.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 248286 19.63 MB/sec execute 552 sec latency 92.065 ms 2024-05-09T07:38:41.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 248521 19.61 MB/sec execute 553 sec latency 110.636 ms 2024-05-09T07:38:42.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 248842 19.60 MB/sec execute 554 sec latency 97.988 ms 2024-05-09T07:38:43.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 249212 19.59 MB/sec execute 555 sec latency 111.257 ms 2024-05-09T07:38:44.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 249804 19.59 MB/sec execute 556 sec latency 61.800 ms 2024-05-09T07:38:45.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 250541 19.64 MB/sec execute 557 sec latency 99.451 ms 2024-05-09T07:38:46.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 251105 19.67 MB/sec execute 558 sec latency 86.516 ms 2024-05-09T07:38:47.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 251483 19.68 MB/sec execute 559 sec latency 217.983 ms 2024-05-09T07:38:48.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 251505 19.65 MB/sec execute 560 sec latency 1218.089 ms 2024-05-09T07:38:49.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 251522 19.62 MB/sec execute 561 sec latency 2218.125 ms 2024-05-09T07:38:50.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 251649 19.60 MB/sec execute 562 sec latency 2651.953 ms 2024-05-09T07:38:51.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 251844 19.58 MB/sec execute 563 sec latency 107.718 ms 2024-05-09T07:38:52.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 252018 19.56 MB/sec execute 564 sec latency 128.027 ms 2024-05-09T07:38:53.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 252124 19.53 MB/sec execute 565 sec latency 144.721 ms 2024-05-09T07:38:54.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 252253 19.51 MB/sec execute 566 sec latency 215.841 ms 2024-05-09T07:38:55.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 252546 19.49 MB/sec execute 567 sec latency 239.902 ms 2024-05-09T07:38:56.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 253009 19.49 MB/sec execute 568 sec latency 135.572 ms 2024-05-09T07:38:57.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 253664 19.52 MB/sec execute 569 sec latency 75.144 ms 2024-05-09T07:38:58.427 INFO:tasks.workunit.client.1.smithi188.stdout: 10 254438 19.56 MB/sec execute 570 sec latency 36.636 ms 2024-05-09T07:38:59.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 254862 19.59 MB/sec execute 571 sec latency 160.706 ms 2024-05-09T07:39:00.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 255253 19.60 MB/sec execute 572 sec latency 85.692 ms 2024-05-09T07:39:01.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 255485 19.59 MB/sec execute 573 sec latency 108.706 ms 2024-05-09T07:39:02.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 255697 19.56 MB/sec execute 574 sec latency 129.590 ms 2024-05-09T07:39:03.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 256014 19.55 MB/sec execute 575 sec latency 133.921 ms 2024-05-09T07:39:04.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 256446 19.54 MB/sec execute 576 sec latency 108.879 ms 2024-05-09T07:39:05.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 257055 19.56 MB/sec execute 577 sec latency 49.418 ms 2024-05-09T07:39:06.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 257745 19.59 MB/sec execute 578 sec latency 65.322 ms 2024-05-09T07:39:07.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 258286 19.63 MB/sec execute 579 sec latency 99.604 ms 2024-05-09T07:39:08.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 258431 19.62 MB/sec execute 580 sec latency 747.363 ms 2024-05-09T07:39:09.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 258448 19.58 MB/sec execute 581 sec latency 1597.296 ms 2024-05-09T07:39:10.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 258627 19.58 MB/sec execute 582 sec latency 992.955 ms 2024-05-09T07:39:11.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 258816 19.56 MB/sec execute 583 sec latency 100.365 ms 2024-05-09T07:39:12.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 259000 19.54 MB/sec execute 584 sec latency 112.327 ms 2024-05-09T07:39:13.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 259181 19.52 MB/sec execute 585 sec latency 155.493 ms 2024-05-09T07:39:14.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 259446 19.50 MB/sec execute 586 sec latency 131.754 ms 2024-05-09T07:39:15.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 259794 19.50 MB/sec execute 587 sec latency 159.530 ms 2024-05-09T07:39:16.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 260382 19.50 MB/sec execute 588 sec latency 131.408 ms 2024-05-09T07:39:17.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 261116 19.54 MB/sec execute 589 sec latency 42.622 ms 2024-05-09T07:39:18.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 261691 19.57 MB/sec execute 590 sec latency 153.940 ms 2024-05-09T07:39:19.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 262146 19.60 MB/sec execute 591 sec latency 73.046 ms 2024-05-09T07:39:20.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 262478 19.60 MB/sec execute 592 sec latency 105.425 ms 2024-05-09T07:39:21.430 INFO:tasks.workunit.client.1.smithi188.stdout: 10 262699 19.58 MB/sec execute 593 sec latency 154.803 ms 2024-05-09T07:39:22.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 262988 19.56 MB/sec execute 594 sec latency 89.553 ms 2024-05-09T07:39:23.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 263399 19.56 MB/sec execute 595 sec latency 94.292 ms 2024-05-09T07:39:24.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 263945 19.55 MB/sec execute 596 sec latency 167.404 ms 2024-05-09T07:39:25.429 INFO:tasks.workunit.client.1.smithi188.stdout: 10 264624 19.60 MB/sec execute 597 sec latency 93.078 ms 2024-05-09T07:39:26.428 INFO:tasks.workunit.client.1.smithi188.stdout: 10 265238 19.63 MB/sec execute 598 sec latency 96.530 ms 2024-05-09T07:39:27.429 INFO:tasks.workunit.client.1.smithi188.stdout: 10 265677 19.65 MB/sec execute 599 sec latency 115.735 ms 2024-05-09T07:39:28.429 INFO:tasks.workunit.client.1.smithi188.stdout: 10 cleanup 600 sec 2024-05-09T07:39:29.429 INFO:tasks.workunit.client.1.smithi188.stdout: 10 cleanup 601 sec 2024-05-09T07:39:30.429 INFO:tasks.workunit.client.1.smithi188.stdout: 10 cleanup 602 sec 2024-05-09T07:39:31.429 INFO:tasks.workunit.client.1.smithi188.stdout: 10 cleanup 603 sec 2024-05-09T07:39:32.356 INFO:tasks.workunit.client.1.smithi188.stdout: 0 cleanup 604 sec 2024-05-09T07:39:32.356 INFO:tasks.workunit.client.1.smithi188.stdout: 2024-05-09T07:39:32.356 INFO:tasks.workunit.client.1.smithi188.stdout: Operation Count AvgLat MaxLat 2024-05-09T07:39:32.356 INFO:tasks.workunit.client.1.smithi188.stdout: ---------------------------------------- 2024-05-09T07:39:32.356 INFO:tasks.workunit.client.1.smithi188.stdout: NTCreateX 377477 6.475 2654.203 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Close 277306 0.003 0.073 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Rename 16004 25.055 1624.004 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Unlink 76127 9.005 687.189 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Qpathinfo 342537 4.729 2690.382 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Qfileinfo 59818 0.001 0.017 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Qfsinfo 62779 0.240 20.806 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Sfileinfo 30680 0.918 310.668 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: Find 132447 0.545 443.245 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: WriteX 186665 0.387 2052.732 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: ReadX 593672 0.083 2681.556 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: LockX 1238 8.678 256.271 2024-05-09T07:39:32.357 INFO:tasks.workunit.client.1.smithi188.stdout: UnlockX 1238 7.652 366.451 2024-05-09T07:39:32.358 INFO:tasks.workunit.client.1.smithi188.stdout: Flush 26391 22.396 3466.819 2024-05-09T07:39:32.358 INFO:tasks.workunit.client.1.smithi188.stdout: 2024-05-09T07:39:32.358 INFO:tasks.workunit.client.1.smithi188.stdout:Throughput 19.6522 MB/sec 10 clients 10 procs max_latency=3466.827 ms 2024-05-09T07:39:32.358 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T07:39:32.358 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-05-09T07:39:34.429 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.1... 2024-05-09T07:39:34.429 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-05-09T07:39:34.778 DEBUG:teuthology.parallel:result is None 2024-05-09T07:39:34.779 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-05-09T07:39:34.812 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-05-09T07:39:34.812 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T07:39:34.812 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:39:34.812 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:39:34.908 DEBUG:teuthology.orchestra.run.smithi188:> set -ex 2024-05-09T07:39:34.908 DEBUG:teuthology.orchestra.run.smithi188:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T07:39:34.965 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:39:35.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.614+0000 7f8c86eda640 1 Processor -- start 2024-05-09T07:39:35.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.614+0000 7f8c86eda640 1 -- start start 2024-05-09T07:39:35.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c86eda640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c80153fb0 0x7f8c8015ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c86eda640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c800a5e50 con 0x7f8c800c2f00 2024-05-09T07:39:35.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c86eda640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c801534d0 con 0x7f8c800c5d80 2024-05-09T07:39:35.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c86eda640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8c80153b20 con 0x7f8c80153fb0 2024-05-09T07:39:35.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c84c76640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f8c800c2f00 0x7f8c801568b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34580/0 (socket says 172.21.15.160:34580) 2024-05-09T07:39:35.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c84c76640 1 -- 172.21.15.160:0/1638800528 learned_addr learned my addr 172.21.15.160:0/1638800528 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:35.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c85477640 1 --2- 172.21.15.160:0/1638800528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c80153fb0 0x7f8c8015ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c85477640 1 -- 172.21.15.160:0/1638800528 >> v1:172.21.15.188:6789/0 conn(0x7f8c800c5d80 legacy=0x7f8c8006b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c85477640 1 -- 172.21.15.160:0/1638800528 >> v1:172.21.15.160:6789/0 conn(0x7f8c800c2f00 legacy=0x7f8c801568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.615+0000 7f8c85477640 1 -- 172.21.15.160:0/1638800528 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c801537b0 con 0x7f8c80153fb0 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.616+0000 7f8c85477640 1 --2- 172.21.15.160:0/1638800528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c80153fb0 0x7f8c8015ec50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009dc0 tx=0x7f8c7802a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab2d2e4b19c27f74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c7f7fe640 1 -- 172.21.15.160:0/1638800528 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78007740 con 0x7f8c80153fb0 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c7f7fe640 1 -- 172.21.15.160:0/1638800528 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c780078e0 con 0x7f8c80153fb0 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c7f7fe640 1 -- 172.21.15.160:0/1638800528 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78007cb0 con 0x7f8c80153fb0 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 -- 172.21.15.160:0/1638800528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c80153fb0 msgr2=0x7f8c8015ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 --2- 172.21.15.160:0/1638800528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c80153fb0 0x7f8c8015ec50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8c78009dc0 tx=0x7f8c7802a5b0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 -- 172.21.15.160:0/1638800528 shutdown_connections 2024-05-09T07:39:35.633 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 --2- 172.21.15.160:0/1638800528 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c80153fb0 0x7f8c8015ec50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 -- 172.21.15.160:0/1638800528 >> 172.21.15.160:0/1638800528 conn(0x7f8c800c7580 msgr2=0x7f8c800a8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 -- 172.21.15.160:0/1638800528 shutdown_connections 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.617+0000 7f8c86eda640 1 -- 172.21.15.160:0/1638800528 wait complete. 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 Processor -- start 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 -- start start 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c800c2f00 0x7f8c800a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c800c5d80 0x7f8c8006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 0x7f8c8006d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8006b9f0 con 0x7f8c80153fb0 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8006b870 con 0x7f8c800c2f00 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.619+0000 7f8c86eda640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f8c800c65b0 con 0x7f8c800c5d80 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 0x7f8c8006d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.634 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 0x7f8c8006d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34588/0 (socket says 172.21.15.160:34588) 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 -- 172.21.15.160:0/3138804907 learned_addr learned my addr 172.21.15.160:0/3138804907 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c84c76640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c800c2f00 0x7f8c800a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c7ffff640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c800c5d80 0x7f8c8006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 -- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c800c5d80 msgr2=0x7f8c8006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c800c5d80 0x7f8c8006d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 -- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c800c2f00 msgr2=0x7f8c800a9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c800c2f00 0x7f8c800a9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 -- 172.21.15.160:0/3138804907 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c800a5900 con 0x7f8c80153fb0 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c84c76640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c800c2f00 0x7f8c800a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c85477640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 0x7f8c8006d800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c7802aba0 tx=0x7f8c78004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:35.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7803f070 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c78031bc0 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c800a5620 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.620+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c78044460 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.621+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c800aaac0 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.621+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c800756c0 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.622+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f8c7803b410 con 0x7f8c80153fb0 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.623+0000 7f8c7d7fa640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c6406be30 0x7f8c6406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.623+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f8c780ac9b0 con 0x7f8c80153fb0 2024-05-09T07:39:35.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.623+0000 7f8c84c76640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c6406be30 0x7f8c6406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.623+0000 7f8c84c76640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c6406be30 0x7f8c6406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005880 tx=0x7f8c74009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:35.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.627+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8c7807a9b0 con 0x7f8c80153fb0 2024-05-09T07:39:35.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.814+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8c8006b300 con 0x7f8c80153fb0 2024-05-09T07:39:35.826 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.815+0000 7f8c7d7fa640 1 -- 172.21.15.160:0/3138804907 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7f8c7807ecb0 con 0x7f8c80153fb0 2024-05-09T07:39:35.826 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:39:35.826 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":27,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:14:26.582005+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:39:35.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c6406be30 msgr2=0x7f8c6406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c6406be30 0x7f8c6406e2f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005880 tx=0x7f8c74009040 comp rx=0 tx=0).stop 2024-05-09T07:39:35.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 msgr2=0x7f8c8006d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 0x7f8c8006d800 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8c7802aba0 tx=0x7f8c78004a40 comp rx=0 tx=0).stop 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 shutdown_connections 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f8c6406be30 0x7f8c6406e2f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c80153fb0 0x7f8c8006d800 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f8c800c5d80 0x7f8c8006d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 --2- 172.21.15.160:0/3138804907 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f8c800c2f00 0x7f8c800a9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 >> 172.21.15.160:0/3138804907 conn(0x7f8c800c7580 msgr2=0x7f8c801512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 shutdown_connections 2024-05-09T07:39:35.829 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.817+0000 7f8c86eda640 1 -- 172.21.15.160:0/3138804907 wait complete. 2024-05-09T07:39:35.841 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T07:13:33.210504+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T07:13:43.238490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T07:13:43.781116+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-09T07:13:47.498198+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-09T07:39:35.842 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T07:39:35.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.936+0000 7f38609c1640 1 Processor -- start 2024-05-09T07:39:35.947 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.936+0000 7f38609c1640 1 -- start start 2024-05-09T07:39:35.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f38609c1640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f385c152a80 0x7f385c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.948 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f38609c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f38609c1640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f385c0c5690 0x7f385c0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f38609c1640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f385c152e60 con 0x7f385c152a80 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f38609c1640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f385c0c80a0 con 0x7f385c0c5690 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f38609c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f385c0c7ae0 con 0x7f385c155e70 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f385c152a80 0x7f385c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f385c152a80 0x7f385c1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34592/0 (socket says 172.21.15.160:34592) 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 -- 172.21.15.160:0/2497460755 learned_addr learned my addr 172.21.15.160:0/2497460755 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f3853fff640 1 --2- 172.21.15.160:0/2497460755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385ad9d640 1 --2- 172.21.15.160:0/2497460755 >> v2:172.21.15.188:3300/0 conn(0x7f385c0c5690 0x7f385c0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 -- 172.21.15.160:0/2497460755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 msgr2=0x7f385c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 --2- 172.21.15.160:0/2497460755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 -- 172.21.15.160:0/2497460755 >> v2:172.21.15.188:3300/0 conn(0x7f385c0c5690 msgr2=0x7f385c0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:35.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 --2- 172.21.15.160:0/2497460755 >> v2:172.21.15.188:3300/0 conn(0x7f385c0c5690 0x7f385c0c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.950 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.937+0000 7f385a59c640 1 -- 172.21.15.160:0/2497460755 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c0c7dc0 con 0x7f385c152a80 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.938+0000 7f385a59c640 1 --2- 172.21.15.160:0/2497460755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c1568e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f385c06d8f0 tx=0x7f384c02a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dfb41a43d4038f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.938+0000 7f3859d9b640 1 -- 172.21.15.160:0/2497460755 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c037070 con 0x7f385c152a80 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.938+0000 7f3859d9b640 1 -- 172.21.15.160:0/2497460755 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f384c02ae00 con 0x7f385c152a80 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.938+0000 7f3859d9b640 1 -- 172.21.15.160:0/2497460755 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c032660 con 0x7f385c152a80 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 -- 172.21.15.160:0/2497460755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 msgr2=0x7f385c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 --2- 172.21.15.160:0/2497460755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c1568e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f385c06d8f0 tx=0x7f384c02a0b0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 -- 172.21.15.160:0/2497460755 shutdown_connections 2024-05-09T07:39:35.953 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 --2- 172.21.15.160:0/2497460755 >> v2:172.21.15.188:3300/0 conn(0x7f385c0c5690 0x7f385c0c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 --2- 172.21.15.160:0/2497460755 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 --2- 172.21.15.160:0/2497460755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c1568e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 -- 172.21.15.160:0/2497460755 >> 172.21.15.160:0/2497460755 conn(0x7f385c072ec0 msgr2=0x7f385c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 -- 172.21.15.160:0/2497460755 shutdown_connections 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.939+0000 7f38609c1640 1 -- 172.21.15.160:0/2497460755 wait complete. 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.940+0000 7f38609c1640 1 Processor -- start 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.940+0000 7f38609c1640 1 -- start start 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f38609c1640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c0c5690 0x7f385c14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f38609c1640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c0c2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f38609c1640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c0c3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f38609c1640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f385c156320 con 0x7f385c152a80 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f38609c1640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f385c158df0 con 0x7f385c0c5690 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f38609c1640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f385c0ad6a0 con 0x7f385c155e70 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c0c2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c0c2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34604/0 (socket says 172.21.15.160:34604) 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 -- 172.21.15.160:0/3562649638 learned_addr learned my addr 172.21.15.160:0/3562649638 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f385a59c640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c0c5690 0x7f385c14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f385ad9d640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c0c3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 -- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 msgr2=0x7f385c0c3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c0c3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 -- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c0c5690 msgr2=0x7f385c14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c0c5690 0x7f385c14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 -- 172.21.15.160:0/3562649638 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f385c1ead20 con 0x7f385c152a80 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3853fff640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c0c2e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f384400d830 tx=0x7f384400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.941+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400b6c0 con 0x7f385c152a80 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.942+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3844019070 con 0x7f385c152a80 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.942+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384400ba80 con 0x7f385c152a80 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.942+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f385c1eafe0 con 0x7f385c152a80 2024-05-09T07:39:35.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.942+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f385c1eb5a0 con 0x7f385c152a80 2024-05-09T07:39:35.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.943+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f385c075650 con 0x7f385c152a80 2024-05-09T07:39:35.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.943+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f384401e460 con 0x7f385c152a80 2024-05-09T07:39:35.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.944+0000 7f3852ffd640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f383c06be30 0x7f383c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:35.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.944+0000 7f385a59c640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f383c06be30 0x7f383c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:35.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.944+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f384408b720 con 0x7f385c152a80 2024-05-09T07:39:35.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.944+0000 7f385a59c640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f383c06be30 0x7f383c06e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f384c034000 tx=0x7f384c02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:35.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:35.948+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3844059720 con 0x7f385c152a80 2024-05-09T07:39:36.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.127+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f385c0ad3a0 con 0x7f385c152a80 2024-05-09T07:39:36.138 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.128+0000 7f3852ffd640 1 -- 172.21.15.160:0/3562649638 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f384405da20 con 0x7f385c152a80 2024-05-09T07:39:36.139 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 1 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f383c06be30 msgr2=0x7f383c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f383c06be30 0x7f383c06e2f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f384c034000 tx=0x7f384c02e000 comp rx=0 tx=0).stop 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 msgr2=0x7f385c0c2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c0c2e30 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f384400d830 tx=0x7f384400dcf0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 shutdown_connections 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f383c06be30 0x7f383c06e2f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f385c155e70 0x7f385c0c3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f385c152a80 0x7f385c0c2e30 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 --2- 172.21.15.160:0/3562649638 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f385c0c5690 0x7f385c14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 >> 172.21.15.160:0/3562649638 conn(0x7f385c072ec0 msgr2=0x7f385c151ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:36.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 shutdown_connections 2024-05-09T07:39:36.143 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.130+0000 7f38609c1640 1 -- 172.21.15.160:0/3562649638 wait complete. 2024-05-09T07:39:36.153 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T07:39:36.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.250+0000 7efeb6b12640 1 Processor -- start 2024-05-09T07:39:36.261 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.250+0000 7efeb6b12640 1 -- start start 2024-05-09T07:39:36.262 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb6b12640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb0150a40 0x7efeb006ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb6b12640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efeb006f7b0 0x7efeb00c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb6b12640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efeb006fe10 con 0x7efeb00c8160 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb6b12640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7efeb00700f0 con 0x7efeb0150a40 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb6b12640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7efeb006baa0 con 0x7efeb006f7b0 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb0150a40 0x7efeb006ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efeb0150a40 0x7efeb006ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34616/0 (socket says 172.21.15.160:34616) 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 -- 172.21.15.160:0/3361179276 learned_addr learned my addr 172.21.15.160:0/3361179276 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeaffff640 1 --2- 172.21.15.160:0/3361179276 >> v2:172.21.15.188:3300/0 conn(0x7efeb006f7b0 0x7efeb00c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 -- 172.21.15.160:0/3361179276 >> v2:172.21.15.188:3300/0 conn(0x7efeb006f7b0 msgr2=0x7efeb00c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 --2- 172.21.15.160:0/3361179276 >> v2:172.21.15.188:3300/0 conn(0x7efeb006f7b0 0x7efeb00c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 -- 172.21.15.160:0/3361179276 >> v1:172.21.15.188:6789/0 conn(0x7efeb00c8160 legacy=0x7efeb0156890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeb48ae640 1 -- 172.21.15.160:0/3361179276 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb006f130 con 0x7efeb0150a40 2024-05-09T07:39:36.263 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.251+0000 7efeaffff640 1 --2- 172.21.15.160:0/3361179276 >> v2:172.21.15.188:3300/0 conn(0x7efeb006f7b0 0x7efeb00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:36.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.252+0000 7efeb48ae640 1 --2- 172.21.15.160:0/3361179276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0150a40 0x7efeb006ebf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efeb006da00 tx=0x7efea402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bef2adc762a35d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.252+0000 7efeaf7fe640 1 -- 172.21.15.160:0/3361179276 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4037070 con 0x7efeb0150a40 2024-05-09T07:39:36.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.252+0000 7efeaf7fe640 1 -- 172.21.15.160:0/3361179276 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efea4032440 con 0x7efeb0150a40 2024-05-09T07:39:36.264 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.252+0000 7efeaf7fe640 1 -- 172.21.15.160:0/3361179276 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea4032810 con 0x7efeb0150a40 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 -- 172.21.15.160:0/3361179276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0150a40 msgr2=0x7efeb006ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 --2- 172.21.15.160:0/3361179276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0150a40 0x7efeb006ebf0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efeb006da00 tx=0x7efea402a270 comp rx=0 tx=0).stop 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 -- 172.21.15.160:0/3361179276 shutdown_connections 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 --2- 172.21.15.160:0/3361179276 >> v2:172.21.15.188:3300/0 conn(0x7efeb006f7b0 0x7efeb00c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 --2- 172.21.15.160:0/3361179276 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0150a40 0x7efeb006ebf0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 -- 172.21.15.160:0/3361179276 >> 172.21.15.160:0/3361179276 conn(0x7efeb006c450 msgr2=0x7efeb006c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 -- 172.21.15.160:0/3361179276 shutdown_connections 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.253+0000 7efeb6b12640 1 -- 172.21.15.160:0/3361179276 wait complete. 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.254+0000 7efeb6b12640 1 Processor -- start 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.254+0000 7efeb6b12640 1 -- start start 2024-05-09T07:39:36.267 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb006f7b0 0x7efeb00c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 0x7efeb014b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb0150a40 0x7efeb014b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efeb0157370 con 0x7efeb00c8160 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efeb01571f0 con 0x7efeb006f7b0 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efeb01574f0 con 0x7efeb0150a40 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 0x7efeb014b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 0x7efeb014b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34630/0 (socket says 172.21.15.160:34630) 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 -- 172.21.15.160:0/566112297 learned_addr learned my addr 172.21.15.160:0/566112297 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb48ae640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb006f7b0 0x7efeb00c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.268 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb50af640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb0150a40 0x7efeb014b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 -- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb0150a40 msgr2=0x7efeb014b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb0150a40 0x7efeb014b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 -- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb006f7b0 msgr2=0x7efeb00c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb006f7b0 0x7efeb00c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 -- 172.21.15.160:0/566112297 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeb014d6d0 con 0x7efeb00c8160 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeaffff640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 0x7efeb014b260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d7d0 tx=0x7efe9c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c0096a0 con 0x7efeb00c8160 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe9c019070 con 0x7efeb00c8160 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c009a80 con 0x7efeb00c8160 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.255+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeb014d9b0 con 0x7efeb00c8160 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.256+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeb014bce0 con 0x7efeb00c8160 2024-05-09T07:39:36.269 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.256+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efeb014ab30 con 0x7efeb00c8160 2024-05-09T07:39:36.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.257+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efe9c009c20 con 0x7efeb00c8160 2024-05-09T07:39:36.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.257+0000 7efead7fa640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efe9406be30 0x7efe9406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.270 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.258+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7efe9c08b480 con 0x7efeb00c8160 2024-05-09T07:39:36.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.258+0000 7efeb48ae640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efe9406be30 0x7efe9406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.271 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.258+0000 7efeb48ae640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efe9406be30 0x7efe9406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efea4034000 tx=0x7efea402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.272 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.262+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efe9c059480 con 0x7efeb00c8160 2024-05-09T07:39:36.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.443+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7efeb0075650 con 0x7efeb00c8160 2024-05-09T07:39:36.454 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.444+0000 7efead7fa640 1 -- 172.21.15.160:0/566112297 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7efe9c05d780 con 0x7efeb00c8160 2024-05-09T07:39:36.454 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efe9406be30 msgr2=0x7efe9406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efe9406be30 0x7efe9406e2f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7efea4034000 tx=0x7efea402e000 comp rx=0 tx=0).stop 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 msgr2=0x7efeb014b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 0x7efeb014b260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7efe9c00d7d0 tx=0x7efe9c00dc90 comp rx=0 tx=0).stop 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 shutdown_connections 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efe9406be30 0x7efe9406e2f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efeb0150a40 0x7efeb014b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00c8160 0x7efeb014b260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 --2- 172.21.15.160:0/566112297 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efeb006f7b0 0x7efeb00c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 >> 172.21.15.160:0/566112297 conn(0x7efeb006c450 msgr2=0x7efeb006c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 shutdown_connections 2024-05-09T07:39:36.457 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.446+0000 7efeb6b12640 1 -- 172.21.15.160:0/566112297 wait complete. 2024-05-09T07:39:36.468 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T07:39:36.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.561+0000 7f71367f9640 1 Processor -- start 2024-05-09T07:39:36.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.561+0000 7f71367f9640 1 -- start start 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f71367f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7130152a80 0x7f71301568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f71367f9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7130155e70 0x7f7130159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f71367f9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f71300c5690 0x7f71300c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f71367f9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f7130152e60 con 0x7f7130155e70 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f71367f9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f71300c80a0 con 0x7f71300c5690 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f71367f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71300c7ae0 con 0x7f7130152a80 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.562+0000 7f7127fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7130155e70 0x7f7130159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f7130155e70 0x7f7130159310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34632/0 (socket says 172.21.15.160:34632) 2024-05-09T07:39:36.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 -- 172.21.15.160:0/3350605118 learned_addr learned my addr 172.21.15.160:0/3350605118 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f712ffff640 1 --2- 172.21.15.160:0/3350605118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7130152a80 0x7f71301568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7134d96640 1 --2- 172.21.15.160:0/3350605118 >> v2:172.21.15.188:3300/0 conn(0x7f71300c5690 0x7f71300c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 -- 172.21.15.160:0/3350605118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7130152a80 msgr2=0x7f71301568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 --2- 172.21.15.160:0/3350605118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7130152a80 0x7f71301568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 -- 172.21.15.160:0/3350605118 >> v2:172.21.15.188:3300/0 conn(0x7f71300c5690 msgr2=0x7f71300c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 --2- 172.21.15.160:0/3350605118 >> v2:172.21.15.188:3300/0 conn(0x7f71300c5690 0x7f71300c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.563+0000 7f7127fff640 1 -- 172.21.15.160:0/3350605118 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71300c7dc0 con 0x7f7130155e70 2024-05-09T07:39:36.575 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f7127fff640 1 --2- 172.21.15.160:0/3350605118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130155e70 0x7f7130159310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71300c1c40 tx=0x7f711802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ba5eb0d4545563 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f712f7fe640 1 -- 172.21.15.160:0/3350605118 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118037070 con 0x7f7130155e70 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f712f7fe640 1 -- 172.21.15.160:0/3350605118 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711802ae00 con 0x7f7130155e70 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f712f7fe640 1 -- 172.21.15.160:0/3350605118 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118032660 con 0x7f7130155e70 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 -- 172.21.15.160:0/3350605118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130155e70 msgr2=0x7f7130159310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 --2- 172.21.15.160:0/3350605118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130155e70 0x7f7130159310 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71300c1c40 tx=0x7f711802a0b0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 -- 172.21.15.160:0/3350605118 shutdown_connections 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 --2- 172.21.15.160:0/3350605118 >> v2:172.21.15.188:3300/0 conn(0x7f71300c5690 0x7f71300c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 --2- 172.21.15.160:0/3350605118 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130155e70 0x7f7130159310 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 --2- 172.21.15.160:0/3350605118 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f7130152a80 0x7f71301568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 -- 172.21.15.160:0/3350605118 >> 172.21.15.160:0/3350605118 conn(0x7f7130072ec0 msgr2=0x7f71300ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:36.579 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 -- 172.21.15.160:0/3350605118 shutdown_connections 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.564+0000 7f71367f9640 1 -- 172.21.15.160:0/3350605118 wait complete. 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 Processor -- start 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 -- start start 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71300c5690 0x7f71300cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 0x7f71300c9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7130155e70 0x7f71300c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f71300ad6a0 con 0x7f7130152a80 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f7130158df0 con 0x7f7130155e70 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f71367f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f71301562f0 con 0x7f71300c5690 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f7127fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 0x7f71300c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f7127fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 0x7f71300c9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34634/0 (socket says 172.21.15.160:34634) 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f7127fff640 1 -- 172.21.15.160:0/2579311248 learned_addr learned my addr 172.21.15.160:0/2579311248 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f7134d96640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7130155e70 0x7f71300c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.566+0000 7f712ffff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71300c5690 0x7f71300cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.580 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f7127fff640 1 -- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71300c5690 msgr2=0x7f71300cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f7127fff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71300c5690 0x7f71300cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f7127fff640 1 -- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7130155e70 msgr2=0x7f71300c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f7127fff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7130155e70 0x7f71300c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f7127fff640 1 -- 172.21.15.160:0/2579311248 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71301eb6b0 con 0x7f7130152a80 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f712ffff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71300c5690 0x7f71300cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f7127fff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 0x7f71300c9390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f711802a0f0 tx=0x7f71180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118041070 con 0x7f7130152a80 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f711803c460 con 0x7f7130152a80 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118037040 con 0x7f7130152a80 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71300b0b90 con 0x7f7130152a80 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.567+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71300cb8f0 con 0x7f7130152a80 2024-05-09T07:39:36.581 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.568+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7130075650 con 0x7f7130152a80 2024-05-09T07:39:36.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.569+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f711803a070 con 0x7f7130152a80 2024-05-09T07:39:36.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.569+0000 7f712d7fa640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f711006be30 0x7f711006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.569+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f71180ab390 con 0x7f7130152a80 2024-05-09T07:39:36.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.569+0000 7f712ffff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f711006be30 0x7f711006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.582 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.570+0000 7f712ffff640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f711006be30 0x7f711006e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7120006fd0 tx=0x7f7120008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.584 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.573+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7118079390 con 0x7f7130152a80 2024-05-09T07:39:36.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.756+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7130151ec0 con 0x7f7130152a80 2024-05-09T07:39:36.767 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.756+0000 7f712d7fa640 1 -- 172.21.15.160:0/2579311248 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f711807d690 con 0x7f7130152a80 2024-05-09T07:39:36.768 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 64 2024-05-09T07:39:36.769 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.758+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f711006be30 msgr2=0x7f711006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.758+0000 7f71367f9640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f711006be30 0x7f711006e2f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7120006fd0 tx=0x7f7120008040 comp rx=0 tx=0).stop 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.758+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 msgr2=0x7f71300c9390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 0x7f71300c9390 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f711802a0f0 tx=0x7f71180046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 shutdown_connections 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f711006be30 0x7f711006e2f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f7130155e70 0x7f71300c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7130152a80 0x7f71300c9390 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 --2- 172.21.15.160:0/2579311248 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f71300c5690 0x7f71300cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 >> 172.21.15.160:0/2579311248 conn(0x7f7130072ec0 msgr2=0x7f7130156aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 shutdown_connections 2024-05-09T07:39:36.770 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.759+0000 7f71367f9640 1 -- 172.21.15.160:0/2579311248 wait complete. 2024-05-09T07:39:36.780 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-09T07:39:36.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.874+0000 7fa8c3516640 1 Processor -- start 2024-05-09T07:39:36.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.874+0000 7fa8c3516640 1 -- start start 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c3516640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8bc0c7d00 0x7fa8bc152110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c3516640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc15c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c3516640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8bc06f930 con 0x7fa8bc0735f0 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c3516640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa8bc0c4210 con 0x7fa8bc0c7d00 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c3516640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc0c8530 con 0x7fa8bc0c80e0 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c12b2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8bc0c7d00 0x7fa8bc152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c0ab1640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7fa8bc0735f0 0x7fa8bc0c3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:44954/0 (socket says 172.21.15.160:44954) 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c0ab1640 1 -- 172.21.15.160:0/2046686016 learned_addr learned my addr 172.21.15.160:0/2046686016 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:36.886 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.875+0000 7fa8c1ab3640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc15c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.876+0000 7fa8c12b2640 1 -- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 msgr2=0x7fa8bc15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.876+0000 7fa8c12b2640 1 --2- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc15c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.876+0000 7fa8c12b2640 1 -- 172.21.15.160:0/2046686016 >> v1:172.21.15.188:6789/0 conn(0x7fa8bc0735f0 legacy=0x7fa8bc0c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.876+0000 7fa8c12b2640 1 -- 172.21.15.160:0/2046686016 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc156230 con 0x7fa8bc0c7d00 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.876+0000 7fa8c12b2640 1 --2- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc152110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0c4390 tx=0x7fa8ac02a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196415ba8e0d131b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8b3fff640 1 -- 172.21.15.160:0/2046686016 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac037070 con 0x7fa8bc0c7d00 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8b3fff640 1 -- 172.21.15.160:0/2046686016 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ac02adb0 con 0x7fa8bc0c7d00 2024-05-09T07:39:36.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8b3fff640 1 -- 172.21.15.160:0/2046686016 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac032660 con 0x7fa8bc0c7d00 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 -- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 msgr2=0x7fa8bc152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 --2- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc152110 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc0c4390 tx=0x7fa8ac02a060 comp rx=0 tx=0).stop 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 -- 172.21.15.160:0/2046686016 shutdown_connections 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 --2- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc15c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 --2- 172.21.15.160:0/2046686016 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc152110 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 -- 172.21.15.160:0/2046686016 >> 172.21.15.160:0/2046686016 conn(0x7fa8bc06c450 msgr2=0x7fa8bc06c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 -- 172.21.15.160:0/2046686016 shutdown_connections 2024-05-09T07:39:36.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.877+0000 7fa8c3516640 1 -- 172.21.15.160:0/2046686016 wait complete. 2024-05-09T07:39:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.878+0000 7fa8c3516640 1 Processor -- start 2024-05-09T07:39:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.878+0000 7fa8c3516640 1 -- start start 2024-05-09T07:39:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c3516640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 0x7fa8bc0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.891 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c3516640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc1485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c3516640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c3516640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc0730a0 con 0x7fa8bc0735f0 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c3516640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8bc156510 con 0x7fa8bc0c7d00 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c3516640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8bc073220 con 0x7fa8bc0c80e0 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c12b2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 0x7fa8bc0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c12b2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 0x7fa8bc0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34636/0 (socket says 172.21.15.160:34636) 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c12b2640 1 -- 172.21.15.160:0/515590193 learned_addr learned my addr 172.21.15.160:0/515590193 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c0ab1640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc1485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.879+0000 7fa8c1ab3640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c12b2640 1 -- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 msgr2=0x7fa8bc148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c12b2640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c12b2640 1 -- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 msgr2=0x7fa8bc1485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:36.892 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c12b2640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc1485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c12b2640 1 -- 172.21.15.160:0/515590193 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8bc1ebdf0 con 0x7fa8bc0735f0 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c12b2640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 0x7fa8bc0ab8d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac000c00 tx=0x7fa8ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac041070 con 0x7fa8bc0735f0 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8ac03c460 con 0x7fa8bc0735f0 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8bc1ec090 con 0x7fa8bc0735f0 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac037040 con 0x7fa8bc0735f0 2024-05-09T07:39:36.893 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.880+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8bc151890 con 0x7fa8bc0735f0 2024-05-09T07:39:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.881+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8bc1e7f10 con 0x7fa8bc0735f0 2024-05-09T07:39:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.882+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8ac044070 con 0x7fa8bc0735f0 2024-05-09T07:39:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.882+0000 7fa8b1ffb640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8a006be30 0x7fa8a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.882+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fa8ac0abbc0 con 0x7fa8bc0735f0 2024-05-09T07:39:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.882+0000 7fa8c0ab1640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8a006be30 0x7fa8a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:36.894 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.883+0000 7fa8c0ab1640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8a006be30 0x7fa8a006e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc14ee70 tx=0x7fa8b8009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:36.897 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:36.887+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8bc1e7f10 con 0x7fa8bc0735f0 2024-05-09T07:39:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.069+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa8bc075650 con 0x7fa8bc0735f0 2024-05-09T07:39:37.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.070+0000 7fa8b1ffb640 1 -- 172.21.15.160:0/515590193 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7fa8ac07a330 con 0x7fa8bc0735f0 2024-05-09T07:39:37.080 INFO:teuthology.orchestra.run.smithi160.stdout:pg_num: 32 2024-05-09T07:39:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8a006be30 msgr2=0x7fa8a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8a006be30 0x7fa8a006e2f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc14ee70 tx=0x7fa8b8009040 comp rx=0 tx=0).stop 2024-05-09T07:39:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 msgr2=0x7fa8bc0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:37.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 0x7fa8bc0ab8d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac000c00 tx=0x7fa8ac0046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 shutdown_connections 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8a006be30 0x7fa8a006e2f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8bc0c7d00 0x7fa8bc1485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8bc0c80e0 0x7fa8bc148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 --2- 172.21.15.160:0/515590193 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8bc0735f0 0x7fa8bc0ab8d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 >> 172.21.15.160:0/515590193 conn(0x7fa8bc06c450 msgr2=0x7fa8bc0c6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 shutdown_connections 2024-05-09T07:39:37.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.072+0000 7fa8c3516640 1 -- 172.21.15.160:0/515590193 wait complete. 2024-05-09T07:39:37.094 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-05-09T07:39:37.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.203+0000 7fa8eb59e640 1 Processor -- start 2024-05-09T07:39:37.214 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.203+0000 7fa8eb59e640 1 -- start start 2024-05-09T07:39:37.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.205+0000 7fa8eb59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec0c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.205+0000 7fa8eb59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa8ec0c47d0 0x7fa8ec15d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.218 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.205+0000 7fa8eb59e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa8ec0c1ce0 0x7fa8ec15d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.205+0000 7fa8eb59e640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec0c5500 con 0x7fa8ec0c47d0 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.205+0000 7fa8eb59e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa8ec0c4c60 con 0x7fa8ec0c1ce0 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.205+0000 7fa8eb59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec0c4f40 con 0x7fa8ec156df0 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.206+0000 7fa8e9d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa8ec0c47d0 0x7fa8ec15d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.206+0000 7fa8e9d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fa8ec0c47d0 0x7fa8ec15d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34652/0 (socket says 172.21.15.160:34652) 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.206+0000 7fa8e9d9b640 1 -- 172.21.15.160:0/3442817129 learned_addr learned my addr 172.21.15.160:0/3442817129 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:37.219 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.206+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/3442817129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec0c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:37.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.211+0000 7fa8ea59c640 1 -- 172.21.15.160:0/3442817129 >> v2:172.21.15.188:3300/0 conn(0x7fa8ec0c1ce0 msgr2=0x7fa8ec15d750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:39:37.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.211+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/3442817129 >> v2:172.21.15.188:3300/0 conn(0x7fa8ec0c1ce0 0x7fa8ec15d750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.211+0000 7fa8ea59c640 1 -- 172.21.15.160:0/3442817129 >> v2:172.21.15.160:3300/0 conn(0x7fa8ec0c47d0 msgr2=0x7fa8ec15d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:37.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.211+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/3442817129 >> v2:172.21.15.160:3300/0 conn(0x7fa8ec0c47d0 0x7fa8ec15d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.223 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.211+0000 7fa8ea59c640 1 -- 172.21.15.160:0/3442817129 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec0c5220 con 0x7fa8ec156df0 2024-05-09T07:39:37.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.213+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/3442817129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec0c5ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0706a0 tx=0x7fa8dc02a0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac71f197a7931a5e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:37.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.214+0000 7fa8e959a640 1 -- 172.21.15.160:0/3442817129 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8dc037070 con 0x7fa8ec156df0 2024-05-09T07:39:37.224 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.214+0000 7fa8e959a640 1 -- 172.21.15.160:0/3442817129 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8dc02ae00 con 0x7fa8ec156df0 2024-05-09T07:39:37.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 -- 172.21.15.160:0/3442817129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 msgr2=0x7fa8ec0c5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:37.225 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/3442817129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec0c5ce0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec0706a0 tx=0x7fa8dc02a0b0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 -- 172.21.15.160:0/3442817129 shutdown_connections 2024-05-09T07:39:37.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/3442817129 >> v2:172.21.15.188:3300/0 conn(0x7fa8ec0c1ce0 0x7fa8ec15d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/3442817129 >> v2:172.21.15.160:3300/0 conn(0x7fa8ec0c47d0 0x7fa8ec15d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/3442817129 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec0c5ce0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.226 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.215+0000 7fa8eb59e640 1 -- 172.21.15.160:0/3442817129 >> 172.21.15.160:0/3442817129 conn(0x7fa8ec0c68d0 msgr2=0x7fa8ec0c6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:37.228 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.217+0000 7fa8eb59e640 1 -- 172.21.15.160:0/3442817129 shutdown_connections 2024-05-09T07:39:37.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.218+0000 7fa8eb59e640 1 -- 172.21.15.160:0/3442817129 wait complete. 2024-05-09T07:39:37.230 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.220+0000 7fa8eb59e640 1 Processor -- start 2024-05-09T07:39:37.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.220+0000 7fa8eb59e640 1 -- start start 2024-05-09T07:39:37.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.221+0000 7fa8eb59e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ec0c1ce0 0x7fa8ec0ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.221+0000 7fa8eb59e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec1f2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.221+0000 7fa8eb59e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 0x7fa8ec0a6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.221+0000 7fa8eb59e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec157350 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.232 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.221+0000 7fa8eb59e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ec1571d0 con 0x7fa8ec0c1ce0 2024-05-09T07:39:37.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.221+0000 7fa8eb59e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ec0c6220 con 0x7fa8ec156df0 2024-05-09T07:39:37.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.222+0000 7fa8ead9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 0x7fa8ec0a6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:37.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.222+0000 7fa8ead9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 0x7fa8ec0a6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34666/0 (socket says 172.21.15.160:34666) 2024-05-09T07:39:37.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.222+0000 7fa8ead9d640 1 -- 172.21.15.160:0/369956230 learned_addr learned my addr 172.21.15.160:0/369956230 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:37.233 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.222+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ec0c1ce0 0x7fa8ec0ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:37.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.225+0000 7fa8ead9d640 1 -- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 msgr2=0x7fa8ec1f2830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T07:39:37.235 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.225+0000 7fa8ead9d640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec1f2830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.225+0000 7fa8ead9d640 1 -- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ec0c1ce0 msgr2=0x7fa8ec0ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:37.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.225+0000 7fa8ead9d640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ec0c1ce0 0x7fa8ec0ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.225+0000 7fa8ead9d640 1 -- 172.21.15.160:0/369956230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ec0a7b50 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.225+0000 7fa8ead9d640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 0x7fa8ec0a6f20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400b9e0 tx=0x7fa8e400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:37.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.226+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4019070 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.236 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.226+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa8e400cb20 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.237 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.226+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4014440 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.228+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ec15a8e0 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.238 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.228+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ec0ab850 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.232+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa8e401c070 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.247 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.232+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ec0755b0 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.246+0000 7fa8daffd640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8cc06bf00 0x7fa8cc06e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:37.257 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.246+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fa8e408af40 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.246+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8cc06bf00 0x7fa8cc06e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:37.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.246+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa8e4059570 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.258 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.247+0000 7fa8ea59c640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8cc06bf00 0x7fa8cc06e3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc034000 tx=0x7fa8dc02e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:37.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.454+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8ec0a9520 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.465 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.455+0000 7fa8daffd640 1 -- 172.21.15.160:0/369956230 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fa8e4058e70 con 0x7fa8ec0ac8e0 2024-05-09T07:39:37.465 INFO:teuthology.orchestra.run.smithi160.stderr:listed 0 entries 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8cc06bf00 msgr2=0x7fa8cc06e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8cc06bf00 0x7fa8cc06e3c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa8dc034000 tx=0x7fa8dc02e000 comp rx=0 tx=0).stop 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 msgr2=0x7fa8ec0a6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 0x7fa8ec0a6f20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400b9e0 tx=0x7fa8e400bea0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 shutdown_connections 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa8cc06bf00 0x7fa8cc06e3c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa8ec0ac8e0 0x7fa8ec0a6f20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa8ec156df0 0x7fa8ec1f2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 --2- 172.21.15.160:0/369956230 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa8ec0c1ce0 0x7fa8ec0ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 >> 172.21.15.160:0/369956230 conn(0x7fa8ec0c68d0 msgr2=0x7fa8ec14f3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 shutdown_connections 2024-05-09T07:39:37.468 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:37.457+0000 7fa8eb59e640 1 -- 172.21.15.160:0/369956230 wait complete. 2024-05-09T07:39:37.479 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-05-09T07:39:37.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:39:37.479 DEBUG:teuthology.orchestra.run.smithi188:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-05-09T07:39:37.611 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:39:37.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:39:37.612 DEBUG:teuthology.orchestra.run.smithi188:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-05-09T07:39:37.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:39:37.673 DEBUG:teuthology.orchestra.run.smithi188:> ip netns list 2024-05-09T07:39:37.939 INFO:teuthology.orchestra.run.smithi188.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-05-09T07:39:37.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:39:37.939 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-05-09T07:39:37.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T07:39:37.966 DEBUG:teuthology.orchestra.run.smithi188:> sudo ip link delete ceph-brx 2024-05-09T07:39:38.062 DEBUG:teuthology.parallel:result is None 2024-05-09T07:39:38.063 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-09T07:39:38.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-09T07:39:38.072 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:39:38.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.164+0000 7f7401ab2640 1 Processor -- start 2024-05-09T07:39:38.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.164+0000 7f7401ab2640 1 -- start start 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f7401ab2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc14f920 0x7f73fc0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f7401ab2640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73fc0c1d20 con 0x7f73fc0719e0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f7401ab2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f73fc0c5b90 con 0x7f73fc0c24f0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f7401ab2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc0c2270 con 0x7f73fc14f920 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fbfff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f73fc0719e0 0x7f73fc0c6230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34592/0 (socket says 172.21.15.160:34592) 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fbfff640 1 -- 172.21.15.160:0/2984347956 learned_addr learned my addr 172.21.15.160:0/2984347956 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fb7fe640 1 --2- 172.21.15.160:0/2984347956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc14f920 0x7f73fc0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1055892108 0 0) 0x7f73fc0c1d20 con 0x7f73fc0719e0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73e4003520 con 0x7f73fc0719e0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3755262468 0 0) 0x7f73fc0c5b90 con 0x7f73fc0c24f0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f73fc0c1d20 con 0x7f73fc0c24f0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fb7fe640 1 -- 172.21.15.160:0/2984347956 >> v1:172.21.15.188:6789/0 conn(0x7f73fc0c24f0 legacy=0x7f73fc1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1114857862 0 0) 0x7f73e4003520 con 0x7f73fc0719e0 2024-05-09T07:39:38.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fb7fe640 1 -- 172.21.15.160:0/2984347956 >> v1:172.21.15.160:6789/0 conn(0x7f73fc0719e0 legacy=0x7f73fc0c6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.165+0000 7f73fb7fe640 1 -- 172.21.15.160:0/2984347956 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc0c1f90 con 0x7f73fc14f920 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.166+0000 7f73fb7fe640 1 --2- 172.21.15.160:0/2984347956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc14f920 0x7f73fc0c3040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73f00097d0 tx=0x7f73f002a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43014dcd8931f13b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.166+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f0037070 con 0x7f73fc14f920 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.166+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73f002ac30 con 0x7f73fc14f920 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.166+0000 7f73fa7fc640 1 -- 172.21.15.160:0/2984347956 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73f40019f0 con 0x7f73fc14f920 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 -- 172.21.15.160:0/2984347956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc14f920 msgr2=0x7f73fc0c3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 --2- 172.21.15.160:0/2984347956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc14f920 0x7f73fc0c3040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f73f00097d0 tx=0x7f73f002a1a0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 -- 172.21.15.160:0/2984347956 shutdown_connections 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 --2- 172.21.15.160:0/2984347956 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc14f920 0x7f73fc0c3040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.179 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 -- 172.21.15.160:0/2984347956 >> 172.21.15.160:0/2984347956 conn(0x7f73fc0a98a0 msgr2=0x7f73fc0a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:38.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 -- 172.21.15.160:0/2984347956 shutdown_connections 2024-05-09T07:39:38.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.167+0000 7f7401ab2640 1 -- 172.21.15.160:0/2984347956 wait complete. 2024-05-09T07:39:38.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.168+0000 7f7401ab2640 1 Processor -- start 2024-05-09T07:39:38.180 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.168+0000 7f7401ab2640 1 -- start start 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f7401ab2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc0719e0 0x7f73fc1e6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f7401ab2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 0x7f73fc14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f7401ab2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73fc14f920 0x7f73fc1527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f7401ab2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc0c3580 con 0x7f73fc0c24f0 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f7401ab2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f73fc14fd00 con 0x7f73fc14f920 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f7401ab2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f73fc06e840 con 0x7f73fc0719e0 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 0x7f73fc14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73fb7fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc0719e0 0x7f73fc1e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 0x7f73fc14be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34680/0 (socket says 172.21.15.160:34680) 2024-05-09T07:39:38.182 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 -- 172.21.15.160:0/1995526573 learned_addr learned my addr 172.21.15.160:0/1995526573 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73fbfff640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73fc14f920 0x7f73fc1527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 -- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc0719e0 msgr2=0x7f73fc1e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc0719e0 0x7f73fc1e6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 -- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73fc14f920 msgr2=0x7f73fc1527d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73fc14f920 0x7f73fc1527d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.169+0000 7f73faffd640 1 -- 172.21.15.160:0/1995526573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73fc1e6020 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.170+0000 7f73faffd640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 0x7f73fc14be90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00c490 tx=0x7f73ec00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.170+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec0052d0 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.170+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f73ec005470 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.170+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73fc06fe40 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.170+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73ec012850 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.170+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73fc150be0 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.171+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73fc06ecd0 con 0x7f73fc0c24f0 2024-05-09T07:39:38.183 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.171+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f73ec0129f0 con 0x7f73fc0c24f0 2024-05-09T07:39:38.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.172+0000 7f73e3fff640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f73e806be30 0x7f73e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.172+0000 7f73fb7fe640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f73e806be30 0x7f73e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.172+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f73ec08adb0 con 0x7f73fc0c24f0 2024-05-09T07:39:38.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.172+0000 7f73fb7fe640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f73e806be30 0x7f73e806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f73f0034000 tx=0x7f73f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.179+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f73fc06ecd0 con 0x7f73fc0c24f0 2024-05-09T07:39:38.392 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.382+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f73fc0756c0 con 0x7f73e806be30 2024-05-09T07:39:38.401 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.391+0000 7f73e3fff640 1 -- 172.21.15.160:0/1995526573 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336227 (secure 0 0 0) 0x7f73fc0756c0 con 0x7f73e806be30 2024-05-09T07:39:38.405 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:39:38.407 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:39:38.409 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f73e806be30 msgr2=0x7f73e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f73e806be30 0x7f73e806e2f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f73f0034000 tx=0x7f73f002e000 comp rx=0 tx=0).stop 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 msgr2=0x7f73fc14be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 0x7f73fc14be90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f73ec00c490 tx=0x7f73ec00c950 comp rx=0 tx=0).stop 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 shutdown_connections 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f73e806be30 0x7f73e806e2f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f73fc14f920 0x7f73fc1527d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73fc0c24f0 0x7f73fc14be90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 --2- 172.21.15.160:0/1995526573 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f73fc0719e0 0x7f73fc1e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 >> 172.21.15.160:0/1995526573 conn(0x7f73fc0a98a0 msgr2=0x7f73fc072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 shutdown_connections 2024-05-09T07:39:38.410 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.395+0000 7f7401ab2640 1 -- 172.21.15.160:0/1995526573 wait complete. 2024-05-09T07:39:38.420 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":802,"stamp":"2024-05-09T07:39:37.561520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310597919,"num_objects":478,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":478,"num_whiteouts":0,"num_read":15563,"num_read_kb":3136874,"num_write":427381,"num_write_kb":32419888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":127,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589252,"ondisk_log_size":589252,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8339872,"kb_used_data":678528,"kb_used_omap":12,"kb_used_meta":7659187,"kb_avail":746634848,"statfs":{"total":773094113280,"available":764554084352,"internally_reserved":0,"allocated":694812672,"data_stored":693388602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":7843008049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,6,4,12,18,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-148056247,"num_objects":-1063,"num_object_clones":0,"num_object_copies":-3364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1063,"num_whiteouts":0,"num_read":84,"num_read_kb":17248,"num_write":3915,"num_write_kb":86342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003252"},"pg_stats":[{"pgid":"2.1f","version":"27'2672","reported_seq":2711,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:51.735149+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:38:51.735149+0000","last_peered":"2024-05-09T07:38:51.735149+0000","last_clean":"2024-05-09T07:38:51.735149+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:38:51.735149+0000","last_undegraded":"2024-05-09T07:38:51.735149+0000","last_fullsized":"2024-05-09T07:38:51.735149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":2684,"num_write_kb":18241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2569","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.929399+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:39:33.929399+0000","last_peered":"2024-05-09T07:39:33.929399+0000","last_clean":"2024-05-09T07:39:33.929399+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:39:33.929399+0000","last_undegraded":"2024-05-09T07:39:33.929399+0000","last_fullsized":"2024-05-09T07:39:33.929399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9807","reported_seq":10237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.981916+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:39:33.981916+0000","last_peered":"2024-05-09T07:39:33.981916+0000","last_clean":"2024-05-09T07:39:33.981916+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:39:33.981916+0000","last_undegraded":"2024-05-09T07:39:33.981916+0000","last_fullsized":"2024-05-09T07:39:33.981916+0000","mapping_epoch":22,"log_start":"27'3504","ondisk_log_start":"27'3504","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6303,"log_dups_size":0,"ondisk_log_size":6303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6872883,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":399,"num_read_kb":101346,"num_write":7160,"num_write_kb":1009567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'7784","reported_seq":7824,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:23.622641+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:39:23.622641+0000","last_peered":"2024-05-09T07:39:23.622641+0000","last_clean":"2024-05-09T07:39:23.622641+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:39:23.622641+0000","last_undegraded":"2024-05-09T07:39:23.622641+0000","last_fullsized":"2024-05-09T07:39:23.622641+0000","mapping_epoch":17,"log_start":"27'1500","ondisk_log_start":"27'1500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":7796,"num_write_kb":54453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2550","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.085746+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:39:34.085746+0000","last_peered":"2024-05-09T07:39:34.085746+0000","last_clean":"2024-05-09T07:39:34.085746+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:39:34.085746+0000","last_undegraded":"2024-05-09T07:39:34.085746+0000","last_fullsized":"2024-05-09T07:39:34.085746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9531","reported_seq":9918,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.127850+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:39:34.127850+0000","last_peered":"2024-05-09T07:39:34.127850+0000","last_clean":"2024-05-09T07:39:34.127850+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:39:34.127850+0000","last_undegraded":"2024-05-09T07:39:34.127850+0000","last_fullsized":"2024-05-09T07:39:34.127850+0000","mapping_epoch":22,"log_start":"27'2329","ondisk_log_start":"27'2329","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6759,"num_write_kb":1095702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9522","reported_seq":9695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.191922+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:39:33.191922+0000","last_peered":"2024-05-09T07:39:33.191922+0000","last_clean":"2024-05-09T07:39:33.191922+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:39:33.191922+0000","last_undegraded":"2024-05-09T07:39:33.191922+0000","last_fullsized":"2024-05-09T07:39:33.191922+0000","mapping_epoch":17,"log_start":"27'2500","ondisk_log_start":"27'2500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7022,"log_dups_size":0,"ondisk_log_size":7022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5211758,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":1083,"num_write":9520,"num_write_kb":77042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2657","reported_seq":2693,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.656593+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:39:33.656593+0000","last_peered":"2024-05-09T07:39:33.656593+0000","last_clean":"2024-05-09T07:39:33.656593+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:39:33.656593+0000","last_undegraded":"2024-05-09T07:39:33.656593+0000","last_fullsized":"2024-05-09T07:39:33.656593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2657,"log_dups_size":0,"ondisk_log_size":2657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9063","reported_seq":9472,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.136390+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:39:34.136390+0000","last_peered":"2024-05-09T07:39:34.136390+0000","last_clean":"2024-05-09T07:39:34.136390+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:39:34.136390+0000","last_undegraded":"2024-05-09T07:39:34.136390+0000","last_fullsized":"2024-05-09T07:39:34.136390+0000","mapping_epoch":22,"log_start":"27'1901","ondisk_log_start":"27'1901","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917224,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6298,"num_write_kb":900735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4288","reported_seq":4325,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.588906+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:39:33.588906+0000","last_peered":"2024-05-09T07:39:33.588906+0000","last_clean":"2024-05-09T07:39:33.588906+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:39:33.588906+0000","last_undegraded":"2024-05-09T07:39:33.588906+0000","last_fullsized":"2024-05-09T07:39:33.588906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":4301,"num_write_kb":47774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2587","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.753943+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:39:33.753943+0000","last_peered":"2024-05-09T07:39:33.753943+0000","last_clean":"2024-05-09T07:39:33.753943+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:39:33.753943+0000","last_undegraded":"2024-05-09T07:39:33.753943+0000","last_fullsized":"2024-05-09T07:39:33.753943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'9248","reported_seq":9721,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.271401+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:39:34.271401+0000","last_peered":"2024-05-09T07:39:34.271401+0000","last_clean":"2024-05-09T07:39:34.271401+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:39:34.271401+0000","last_undegraded":"2024-05-09T07:39:34.271401+0000","last_fullsized":"2024-05-09T07:39:34.271401+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":442,"num_read_kb":112268,"num_write":6537,"num_write_kb":853848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2578","reported_seq":2615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.930484+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:39:33.930484+0000","last_peered":"2024-05-09T07:39:33.930484+0000","last_clean":"2024-05-09T07:39:33.930484+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:39:33.930484+0000","last_undegraded":"2024-05-09T07:39:33.930484+0000","last_fullsized":"2024-05-09T07:39:33.930484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6961","reported_seq":6999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:03.399820+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:39:03.399820+0000","last_peered":"2024-05-09T07:39:03.399820+0000","last_clean":"2024-05-09T07:39:03.399820+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:39:03.399820+0000","last_undegraded":"2024-05-09T07:39:03.399820+0000","last_fullsized":"2024-05-09T07:39:03.399820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6961,"log_dups_size":0,"ondisk_log_size":6961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6969,"num_write_kb":53856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9806","reported_seq":10053,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.172117+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:39:34.172117+0000","last_peered":"2024-05-09T07:39:34.172117+0000","last_clean":"2024-05-09T07:39:34.172117+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:39:34.172117+0000","last_undegraded":"2024-05-09T07:39:34.172117+0000","last_fullsized":"2024-05-09T07:39:34.172117+0000","mapping_epoch":22,"log_start":"27'2633","ondisk_log_start":"27'2633","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":54610,"num_write":7039,"num_write_kb":1008697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2599","reported_seq":2636,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.034015+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:39:34.034015+0000","last_peered":"2024-05-09T07:39:34.034015+0000","last_clean":"2024-05-09T07:39:34.034015+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:39:34.034015+0000","last_undegraded":"2024-05-09T07:39:34.034015+0000","last_fullsized":"2024-05-09T07:39:34.034015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7778","reported_seq":7817,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:42.793042+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:38:42.793042+0000","last_peered":"2024-05-09T07:38:42.793042+0000","last_clean":"2024-05-09T07:38:42.793042+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:38:42.793042+0000","last_undegraded":"2024-05-09T07:38:42.793042+0000","last_fullsized":"2024-05-09T07:38:42.793042+0000","mapping_epoch":17,"log_start":"27'400","ondisk_log_start":"27'400","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":58435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9627","reported_seq":10167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.260189+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:39:34.260189+0000","last_peered":"2024-05-09T07:39:34.260189+0000","last_clean":"2024-05-09T07:39:34.260189+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:39:34.260189+0000","last_undegraded":"2024-05-09T07:39:34.260189+0000","last_fullsized":"2024-05-09T07:39:34.260189+0000","mapping_epoch":22,"log_start":"27'2443","ondisk_log_start":"27'2443","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2628189,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":508,"num_read_kb":129802,"num_write":6847,"num_write_kb":970185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2672","reported_seq":2709,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.851191+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:39:33.851191+0000","last_peered":"2024-05-09T07:39:33.851191+0000","last_clean":"2024-05-09T07:39:33.851191+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:39:33.851191+0000","last_undegraded":"2024-05-09T07:39:33.851191+0000","last_fullsized":"2024-05-09T07:39:33.851191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3774","reported_seq":3813,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:40.266938+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:37:40.266938+0000","last_peered":"2024-05-09T07:37:40.266938+0000","last_clean":"2024-05-09T07:37:40.266938+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:37:40.266938+0000","last_undegraded":"2024-05-09T07:37:40.266938+0000","last_fullsized":"2024-05-09T07:37:40.266938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3774,"log_dups_size":0,"ondisk_log_size":3774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3771,"num_write_kb":27117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9040","reported_seq":9652,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.430298+0000","last_change":"2024-05-09T07:14:26.646797+0000","last_active":"2024-05-09T07:39:33.430298+0000","last_peered":"2024-05-09T07:39:33.430298+0000","last_clean":"2024-05-09T07:39:33.430298+0000","last_became_active":"2024-05-09T07:14:26.646222+0000","last_became_peered":"2024-05-09T07:14:26.646222+0000","last_unstale":"2024-05-09T07:39:33.430298+0000","last_undegraded":"2024-05-09T07:39:33.430298+0000","last_fullsized":"2024-05-09T07:39:33.430298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3553201,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6364,"num_write_kb":893211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2580","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.213138+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:39:34.213138+0000","last_peered":"2024-05-09T07:39:34.213138+0000","last_clean":"2024-05-09T07:39:34.213138+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:39:34.213138+0000","last_undegraded":"2024-05-09T07:39:34.213138+0000","last_fullsized":"2024-05-09T07:39:34.213138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2263","reported_seq":2301,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:10.954485+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:37:10.954485+0000","last_peered":"2024-05-09T07:37:10.954485+0000","last_clean":"2024-05-09T07:37:10.954485+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:37:10.954485+0000","last_undegraded":"2024-05-09T07:37:10.954485+0000","last_fullsized":"2024-05-09T07:37:10.954485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2272,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9928","reported_seq":10064,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.216405+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:39:34.216405+0000","last_peered":"2024-05-09T07:39:34.216405+0000","last_clean":"2024-05-09T07:39:34.216405+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:39:34.216405+0000","last_undegraded":"2024-05-09T07:39:34.216405+0000","last_fullsized":"2024-05-09T07:39:34.216405+0000","mapping_epoch":22,"log_start":"27'2728","ondisk_log_start":"27'2728","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1371263,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":26416,"num_write":7233,"num_write_kb":974347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.954387+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:39:33.954387+0000","last_peered":"2024-05-09T07:39:33.954387+0000","last_clean":"2024-05-09T07:39:33.954387+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:39:33.954387+0000","last_undegraded":"2024-05-09T07:39:33.954387+0000","last_fullsized":"2024-05-09T07:39:33.954387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4066","reported_seq":4105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:39.865043+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:38:39.865043+0000","last_peered":"2024-05-09T07:38:39.865043+0000","last_clean":"2024-05-09T07:38:39.865043+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:38:39.865043+0000","last_undegraded":"2024-05-09T07:38:39.865043+0000","last_fullsized":"2024-05-09T07:38:39.865043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":39474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9245","reported_seq":9663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.248822+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:39:34.248822+0000","last_peered":"2024-05-09T07:39:34.248822+0000","last_clean":"2024-05-09T07:39:34.248822+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:39:34.248822+0000","last_undegraded":"2024-05-09T07:39:34.248822+0000","last_fullsized":"2024-05-09T07:39:34.248822+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7223,"log_dups_size":0,"ondisk_log_size":7223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3175846,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":97790,"num_write":6533,"num_write_kb":843792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2528","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.812065+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:39:33.812065+0000","last_peered":"2024-05-09T07:39:33.812065+0000","last_clean":"2024-05-09T07:39:33.812065+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:39:33.812065+0000","last_undegraded":"2024-05-09T07:39:33.812065+0000","last_fullsized":"2024-05-09T07:39:33.812065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6992","reported_seq":7638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.782270+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:39:33.782270+0000","last_peered":"2024-05-09T07:39:33.782270+0000","last_clean":"2024-05-09T07:39:33.782270+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:39:33.782270+0000","last_undegraded":"2024-05-09T07:39:33.782270+0000","last_fullsized":"2024-05-09T07:39:33.782270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6992,"log_dups_size":0,"ondisk_log_size":6992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":608,"num_read_kb":4487,"num_write":9811,"num_write_kb":61141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9147","reported_seq":9330,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.892317+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:39:33.892317+0000","last_peered":"2024-05-09T07:39:33.892317+0000","last_clean":"2024-05-09T07:39:33.892317+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:39:33.892317+0000","last_undegraded":"2024-05-09T07:39:33.892317+0000","last_fullsized":"2024-05-09T07:39:33.892317+0000","mapping_epoch":22,"log_start":"27'2834","ondisk_log_start":"27'2834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1591544,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":150,"num_read_kb":38100,"num_write":6423,"num_write_kb":884596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2636","reported_seq":2674,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.131081+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:39:34.131081+0000","last_peered":"2024-05-09T07:39:34.131081+0000","last_clean":"2024-05-09T07:39:34.131081+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:39:34.131081+0000","last_undegraded":"2024-05-09T07:39:34.131081+0000","last_fullsized":"2024-05-09T07:39:34.131081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2636,"log_dups_size":0,"ondisk_log_size":2636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6519","reported_seq":6558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:29.207111+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:39:29.207111+0000","last_peered":"2024-05-09T07:39:29.207111+0000","last_clean":"2024-05-09T07:39:29.207111+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:39:29.207111+0000","last_undegraded":"2024-05-09T07:39:29.207111+0000","last_fullsized":"2024-05-09T07:39:29.207111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6519,"log_dups_size":0,"ondisk_log_size":6519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6522,"num_write_kb":53434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9269","reported_seq":10155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.551978+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:39:34.551978+0000","last_peered":"2024-05-09T07:39:34.551978+0000","last_clean":"2024-05-09T07:39:34.551978+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:39:34.551978+0000","last_undegraded":"2024-05-09T07:39:34.551978+0000","last_fullsized":"2024-05-09T07:39:34.551978+0000","mapping_epoch":22,"log_start":"27'1856","ondisk_log_start":"27'1856","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260096,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":853,"num_read_kb":216662,"num_write":6473,"num_write_kb":849337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2515","reported_seq":2537,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.642256+0000","last_change":"2024-05-09T07:14:26.645955+0000","last_active":"2024-05-09T07:39:33.642256+0000","last_peered":"2024-05-09T07:39:33.642256+0000","last_clean":"2024-05-09T07:39:33.642256+0000","last_became_active":"2024-05-09T07:14:26.645530+0000","last_became_peered":"2024-05-09T07:14:26.645530+0000","last_unstale":"2024-05-09T07:39:33.642256+0000","last_undegraded":"2024-05-09T07:39:33.642256+0000","last_fullsized":"2024-05-09T07:39:33.642256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'8371","reported_seq":8411,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:26.913891+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:39:26.913891+0000","last_peered":"2024-05-09T07:39:26.913891+0000","last_clean":"2024-05-09T07:39:26.913891+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:39:26.913891+0000","last_undegraded":"2024-05-09T07:39:26.913891+0000","last_fullsized":"2024-05-09T07:39:26.913891+0000","mapping_epoch":17,"log_start":"27'1000","ondisk_log_start":"27'1000","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8366,"num_write_kb":75026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9858","reported_seq":10329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.561712+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:39:34.561712+0000","last_peered":"2024-05-09T07:39:34.561712+0000","last_clean":"2024-05-09T07:39:34.561712+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:39:34.561712+0000","last_undegraded":"2024-05-09T07:39:34.561712+0000","last_fullsized":"2024-05-09T07:39:34.561712+0000","mapping_epoch":22,"log_start":"27'2476","ondisk_log_start":"27'2476","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":438,"num_read_kb":111252,"num_write":7195,"num_write_kb":963969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2563","reported_seq":2600,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.119968+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:39:34.119968+0000","last_peered":"2024-05-09T07:39:34.119968+0000","last_clean":"2024-05-09T07:39:34.119968+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:39:34.119968+0000","last_undegraded":"2024-05-09T07:39:34.119968+0000","last_fullsized":"2024-05-09T07:39:34.119968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2563,"log_dups_size":0,"ondisk_log_size":2563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'9194","reported_seq":9234,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:20.730299+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:39:20.730299+0000","last_peered":"2024-05-09T07:39:20.730299+0000","last_clean":"2024-05-09T07:39:20.730299+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:39:20.730299+0000","last_undegraded":"2024-05-09T07:39:20.730299+0000","last_fullsized":"2024-05-09T07:39:20.730299+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9195,"num_write_kb":63563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9946","reported_seq":10111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.331272+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:39:34.331272+0000","last_peered":"2024-05-09T07:39:34.331272+0000","last_clean":"2024-05-09T07:39:34.331272+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:39:34.331272+0000","last_undegraded":"2024-05-09T07:39:34.331272+0000","last_fullsized":"2024-05-09T07:39:34.331272+0000","mapping_epoch":22,"log_start":"27'2616","ondisk_log_start":"27'2616","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":33528,"num_write":7175,"num_write_kb":934104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2576","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.026717+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:39:34.026717+0000","last_peered":"2024-05-09T07:39:34.026717+0000","last_clean":"2024-05-09T07:39:34.026717+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:39:34.026717+0000","last_undegraded":"2024-05-09T07:39:34.026717+0000","last_fullsized":"2024-05-09T07:39:34.026717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7786","reported_seq":8725,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.660101+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:39:33.660101+0000","last_peered":"2024-05-09T07:39:33.660101+0000","last_clean":"2024-05-09T07:39:33.660101+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:39:33.660101+0000","last_undegraded":"2024-05-09T07:39:33.660101+0000","last_fullsized":"2024-05-09T07:39:33.660101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7786,"log_dups_size":0,"ondisk_log_size":7786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8410861,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":900,"num_read_kb":4675,"num_write":7771,"num_write_kb":62460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9564","reported_seq":9939,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.665427+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:39:34.665427+0000","last_peered":"2024-05-09T07:39:34.665427+0000","last_clean":"2024-05-09T07:39:34.665427+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:39:34.665427+0000","last_undegraded":"2024-05-09T07:39:34.665427+0000","last_fullsized":"2024-05-09T07:39:34.665427+0000","mapping_epoch":22,"log_start":"27'2181","ondisk_log_start":"27'2181","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":87122,"num_write":6858,"num_write_kb":899446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2521","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.239147+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:39:34.239147+0000","last_peered":"2024-05-09T07:39:34.239147+0000","last_clean":"2024-05-09T07:39:34.239147+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:39:34.239147+0000","last_undegraded":"2024-05-09T07:39:34.239147+0000","last_fullsized":"2024-05-09T07:39:34.239147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7846","reported_seq":7885,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:57.039762+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:38:57.039762+0000","last_peered":"2024-05-09T07:38:57.039762+0000","last_clean":"2024-05-09T07:38:57.039762+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:38:57.039762+0000","last_undegraded":"2024-05-09T07:38:57.039762+0000","last_fullsized":"2024-05-09T07:38:57.039762+0000","mapping_epoch":17,"log_start":"27'800","ondisk_log_start":"27'800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7848,"num_write_kb":58481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9410","reported_seq":9664,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.339794+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:39:34.339794+0000","last_peered":"2024-05-09T07:39:34.339794+0000","last_clean":"2024-05-09T07:39:34.339794+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:39:34.339794+0000","last_undegraded":"2024-05-09T07:39:34.339794+0000","last_fullsized":"2024-05-09T07:39:34.339794+0000","mapping_epoch":22,"log_start":"27'2042","ondisk_log_start":"27'2042","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":56388,"num_write":6716,"num_write_kb":1113422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2631","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.545182+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:39:33.545182+0000","last_peered":"2024-05-09T07:39:33.545182+0000","last_clean":"2024-05-09T07:39:33.545182+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:39:33.545182+0000","last_undegraded":"2024-05-09T07:39:33.545182+0000","last_fullsized":"2024-05-09T07:39:33.545182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2631,"log_dups_size":0,"ondisk_log_size":2631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4648","reported_seq":4688,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:53.493345+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:38:53.493345+0000","last_peered":"2024-05-09T07:38:53.493345+0000","last_clean":"2024-05-09T07:38:53.493345+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:38:53.493345+0000","last_undegraded":"2024-05-09T07:38:53.493345+0000","last_fullsized":"2024-05-09T07:38:53.493345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4648,"log_dups_size":0,"ondisk_log_size":4648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4652,"num_write_kb":39916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:35:39.524573+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:35:39.524573+0000","last_peered":"2024-05-09T07:35:39.524573+0000","last_clean":"2024-05-09T07:35:39.524573+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:35:39.524573+0000","last_undegraded":"2024-05-09T07:35:39.524573+0000","last_fullsized":"2024-05-09T07:35:39.524573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8966","reported_seq":9586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.979598+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:39:33.979598+0000","last_peered":"2024-05-09T07:39:33.979598+0000","last_clean":"2024-05-09T07:39:33.979598+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:39:33.979598+0000","last_undegraded":"2024-05-09T07:39:33.979598+0000","last_fullsized":"2024-05-09T07:39:33.979598+0000","mapping_epoch":22,"log_start":"27'1027","ondisk_log_start":"27'1027","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7939,"log_dups_size":0,"ondisk_log_size":7939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524831,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":587,"num_read_kb":149098,"num_write":6329,"num_write_kb":900398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2553","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.175294+0000","last_change":"2024-05-09T07:14:26.591809+0000","last_active":"2024-05-09T07:39:33.175294+0000","last_peered":"2024-05-09T07:39:33.175294+0000","last_clean":"2024-05-09T07:39:33.175294+0000","last_became_active":"2024-05-09T07:14:26.591447+0000","last_became_peered":"2024-05-09T07:14:26.591447+0000","last_unstale":"2024-05-09T07:39:33.175294+0000","last_undegraded":"2024-05-09T07:39:33.175294+0000","last_fullsized":"2024-05-09T07:39:33.175294+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"27'7870","reported_seq":7908,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:33.438472+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:38:33.438472+0000","last_peered":"2024-05-09T07:38:33.438472+0000","last_clean":"2024-05-09T07:38:33.438472+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:38:33.438472+0000","last_undegraded":"2024-05-09T07:38:33.438472+0000","last_fullsized":"2024-05-09T07:38:33.438472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7870,"log_dups_size":0,"ondisk_log_size":7870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7874,"num_write_kb":58494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9727","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.003249+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:39:34.003249+0000","last_peered":"2024-05-09T07:39:34.003249+0000","last_clean":"2024-05-09T07:39:34.003249+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:39:34.003249+0000","last_undegraded":"2024-05-09T07:39:34.003249+0000","last_fullsized":"2024-05-09T07:39:34.003249+0000","mapping_epoch":22,"log_start":"27'1734","ondisk_log_start":"27'1734","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7993,"log_dups_size":0,"ondisk_log_size":7993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976481,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":114046,"num_write":7013,"num_write_kb":962272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2581","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.572472+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:39:33.572472+0000","last_peered":"2024-05-09T07:39:33.572472+0000","last_clean":"2024-05-09T07:39:33.572472+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:39:33.572472+0000","last_undegraded":"2024-05-09T07:39:33.572472+0000","last_fullsized":"2024-05-09T07:39:33.572472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2581,"log_dups_size":0,"ondisk_log_size":2581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7541","reported_seq":7580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.569151+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:39:17.569151+0000","last_peered":"2024-05-09T07:39:17.569151+0000","last_clean":"2024-05-09T07:39:17.569151+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:39:17.569151+0000","last_undegraded":"2024-05-09T07:39:17.569151+0000","last_fullsized":"2024-05-09T07:39:17.569151+0000","mapping_epoch":17,"log_start":"27'200","ondisk_log_start":"27'200","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7542,"num_write_kb":58281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9828","reported_seq":10111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.629301+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:39:34.629301+0000","last_peered":"2024-05-09T07:39:34.629301+0000","last_clean":"2024-05-09T07:39:34.629301+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:39:34.629301+0000","last_undegraded":"2024-05-09T07:39:34.629301+0000","last_fullsized":"2024-05-09T07:39:34.629301+0000","mapping_epoch":22,"log_start":"27'2499","ondisk_log_start":"27'2499","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7329,"log_dups_size":0,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260096,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":251,"num_read_kb":63754,"num_write":7055,"num_write_kb":893799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2449","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.207708+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:39:34.207708+0000","last_peered":"2024-05-09T07:39:34.207708+0000","last_clean":"2024-05-09T07:39:34.207708+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:39:34.207708+0000","last_undegraded":"2024-05-09T07:39:34.207708+0000","last_fullsized":"2024-05-09T07:39:34.207708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8861","reported_seq":8899,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:36.701798+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:38:36.701798+0000","last_peered":"2024-05-09T07:38:36.701798+0000","last_clean":"2024-05-09T07:38:36.701798+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:38:36.701798+0000","last_undegraded":"2024-05-09T07:38:36.701798+0000","last_fullsized":"2024-05-09T07:38:36.701798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8861,"log_dups_size":0,"ondisk_log_size":8861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8852,"num_write_kb":79483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9353","reported_seq":10368,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.782422+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:39:33.782422+0000","last_peered":"2024-05-09T07:39:33.782422+0000","last_clean":"2024-05-09T07:39:33.782422+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:39:33.782422+0000","last_undegraded":"2024-05-09T07:39:33.782422+0000","last_fullsized":"2024-05-09T07:39:33.782422+0000","mapping_epoch":22,"log_start":"27'1982","ondisk_log_start":"27'1982","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4053700,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":983,"num_read_kb":249682,"num_write":6557,"num_write_kb":1037240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2553","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.169185+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:39:34.169185+0000","last_peered":"2024-05-09T07:39:34.169185+0000","last_clean":"2024-05-09T07:39:34.169185+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:39:34.169185+0000","last_undegraded":"2024-05-09T07:39:34.169185+0000","last_fullsized":"2024-05-09T07:39:34.169185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5464","reported_seq":5502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:48.455777+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:38:48.455777+0000","last_peered":"2024-05-09T07:38:48.455777+0000","last_clean":"2024-05-09T07:38:48.455777+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:38:48.455777+0000","last_undegraded":"2024-05-09T07:38:48.455777+0000","last_fullsized":"2024-05-09T07:38:48.455777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5457,"num_write_kb":44489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9303","reported_seq":9861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.254376+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:39:34.254376+0000","last_peered":"2024-05-09T07:39:34.254376+0000","last_clean":"2024-05-09T07:39:34.254376+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:39:34.254376+0000","last_undegraded":"2024-05-09T07:39:34.254376+0000","last_fullsized":"2024-05-09T07:39:34.254376+0000","mapping_epoch":22,"log_start":"27'2103","ondisk_log_start":"27'2103","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":527,"num_read_kb":133858,"num_write":6611,"num_write_kb":868619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2614","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.416883+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:39:34.416883+0000","last_peered":"2024-05-09T07:39:34.416883+0000","last_clean":"2024-05-09T07:39:34.416883+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:39:34.416883+0000","last_undegraded":"2024-05-09T07:39:34.416883+0000","last_fullsized":"2024-05-09T07:39:34.416883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2614,"log_dups_size":0,"ondisk_log_size":2614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3463","reported_seq":3501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:20.735079+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:39:20.735079+0000","last_peered":"2024-05-09T07:39:20.735079+0000","last_clean":"2024-05-09T07:39:20.735079+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:39:20.735079+0000","last_undegraded":"2024-05-09T07:39:20.735079+0000","last_fullsized":"2024-05-09T07:39:20.735079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3463,"log_dups_size":0,"ondisk_log_size":3463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3477,"num_write_kb":39113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9477","reported_seq":10052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.565062+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:39:34.565062+0000","last_peered":"2024-05-09T07:39:34.565062+0000","last_clean":"2024-05-09T07:39:34.565062+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:39:34.565062+0000","last_undegraded":"2024-05-09T07:39:34.565062+0000","last_fullsized":"2024-05-09T07:39:34.565062+0000","mapping_epoch":22,"log_start":"27'2114","ondisk_log_start":"27'2114","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":138176,"num_write":6755,"num_write_kb":1181456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2485","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.378977+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:39:33.378977+0000","last_peered":"2024-05-09T07:39:33.378977+0000","last_clean":"2024-05-09T07:39:33.378977+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:39:33.378977+0000","last_undegraded":"2024-05-09T07:39:33.378977+0000","last_fullsized":"2024-05-09T07:39:33.378977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4963","reported_seq":5002,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.760052+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:39:17.760052+0000","last_peered":"2024-05-09T07:39:17.760052+0000","last_clean":"2024-05-09T07:39:17.760052+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:39:17.760052+0000","last_undegraded":"2024-05-09T07:39:17.760052+0000","last_fullsized":"2024-05-09T07:39:17.760052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4963,"log_dups_size":0,"ondisk_log_size":4963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":4974,"num_write_kb":44143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9304","reported_seq":9607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.999932+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:39:33.999932+0000","last_peered":"2024-05-09T07:39:33.999932+0000","last_clean":"2024-05-09T07:39:33.999932+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:39:33.999932+0000","last_undegraded":"2024-05-09T07:39:33.999932+0000","last_fullsized":"2024-05-09T07:39:33.999932+0000","mapping_epoch":22,"log_start":"27'3038","ondisk_log_start":"27'3038","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6266,"log_dups_size":0,"ondisk_log_size":6266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4623858,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":272,"num_read_kb":69088,"num_write":6503,"num_write_kb":924175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2595","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.358643+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:39:34.358643+0000","last_peered":"2024-05-09T07:39:34.358643+0000","last_clean":"2024-05-09T07:39:34.358643+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:39:34.358643+0000","last_undegraded":"2024-05-09T07:39:34.358643+0000","last_fullsized":"2024-05-09T07:39:34.358643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2649","reported_seq":2688,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:14.327549+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:39:14.327549+0000","last_peered":"2024-05-09T07:39:14.327549+0000","last_clean":"2024-05-09T07:39:14.327549+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:39:14.327549+0000","last_undegraded":"2024-05-09T07:39:14.327549+0000","last_fullsized":"2024-05-09T07:39:14.327549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2650,"num_write_kb":30266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9217","reported_seq":9444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.929998+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:39:33.929998+0000","last_peered":"2024-05-09T07:39:33.929998+0000","last_clean":"2024-05-09T07:39:33.929998+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:39:33.929998+0000","last_undegraded":"2024-05-09T07:39:33.929998+0000","last_fullsized":"2024-05-09T07:39:33.929998+0000","mapping_epoch":22,"log_start":"27'2939","ondisk_log_start":"27'2939","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6278,"log_dups_size":0,"ondisk_log_size":6278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519806,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":197,"num_read_kb":50038,"num_write":6657,"num_write_kb":910804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2552","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.373769+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:39:34.373769+0000","last_peered":"2024-05-09T07:39:34.373769+0000","last_clean":"2024-05-09T07:39:34.373769+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:39:34.373769+0000","last_undegraded":"2024-05-09T07:39:34.373769+0000","last_fullsized":"2024-05-09T07:39:34.373769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2552,"log_dups_size":0,"ondisk_log_size":2552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6995","reported_seq":7033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:29.888509+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:39:29.888509+0000","last_peered":"2024-05-09T07:39:29.888509+0000","last_clean":"2024-05-09T07:39:29.888509+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:39:29.888509+0000","last_undegraded":"2024-05-09T07:39:29.888509+0000","last_fullsized":"2024-05-09T07:39:29.888509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6995,"log_dups_size":0,"ondisk_log_size":6995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6987,"num_write_kb":57820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9727","reported_seq":9995,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.789422+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:39:33.789422+0000","last_peered":"2024-05-09T07:39:33.789422+0000","last_clean":"2024-05-09T07:39:33.789422+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:39:33.789422+0000","last_undegraded":"2024-05-09T07:39:33.789422+0000","last_fullsized":"2024-05-09T07:39:33.789422+0000","mapping_epoch":22,"log_start":"27'2359","ondisk_log_start":"27'2359","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2496915,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":237,"num_read_kb":60198,"num_write":7040,"num_write_kb":1050276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2497","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.331398+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:39:33.331398+0000","last_peered":"2024-05-09T07:39:33.331398+0000","last_clean":"2024-05-09T07:39:33.331398+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:39:33.331398+0000","last_undegraded":"2024-05-09T07:39:33.331398+0000","last_fullsized":"2024-05-09T07:39:33.331398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10540","reported_seq":11205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.746477+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:39:17.746477+0000","last_peered":"2024-05-09T07:39:17.746477+0000","last_clean":"2024-05-09T07:39:17.746477+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:39:17.746477+0000","last_undegraded":"2024-05-09T07:39:17.746477+0000","last_fullsized":"2024-05-09T07:39:17.746477+0000","mapping_epoch":17,"log_start":"27'2600","ondisk_log_start":"27'2600","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7940,"log_dups_size":0,"ondisk_log_size":7940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":627,"num_read_kb":4520,"num_write":10533,"num_write_kb":76795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9186","reported_seq":9429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.849229+0000","last_change":"2024-05-09T07:14:26.592176+0000","last_active":"2024-05-09T07:39:33.849229+0000","last_peered":"2024-05-09T07:39:33.849229+0000","last_clean":"2024-05-09T07:39:33.849229+0000","last_became_active":"2024-05-09T07:14:26.590721+0000","last_became_peered":"2024-05-09T07:14:26.590721+0000","last_unstale":"2024-05-09T07:39:33.849229+0000","last_undegraded":"2024-05-09T07:39:33.849229+0000","last_fullsized":"2024-05-09T07:39:33.849229+0000","mapping_epoch":26,"log_start":"27'1828","ondisk_log_start":"27'1828","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:38.451332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4278543,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":221,"num_read_kb":56134,"num_write":6500,"num_write_kb":978701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"27'2569","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.706846+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:39:33.706846+0000","last_peered":"2024-05-09T07:39:33.706846+0000","last_clean":"2024-05-09T07:39:33.706846+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:39:33.706846+0000","last_undegraded":"2024-05-09T07:39:33.706846+0000","last_fullsized":"2024-05-09T07:39:33.706846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6174","reported_seq":6212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:27.333611+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:39:27.333611+0000","last_peered":"2024-05-09T07:39:27.333611+0000","last_clean":"2024-05-09T07:39:27.333611+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:39:27.333611+0000","last_undegraded":"2024-05-09T07:39:27.333611+0000","last_fullsized":"2024-05-09T07:39:27.333611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6180,"num_write_kb":41394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9167","reported_seq":9510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.231279+0000","last_change":"2024-05-09T07:14:26.596397+0000","last_active":"2024-05-09T07:39:34.231279+0000","last_peered":"2024-05-09T07:39:34.231279+0000","last_clean":"2024-05-09T07:39:34.231279+0000","last_became_active":"2024-05-09T07:14:26.595936+0000","last_became_peered":"2024-05-09T07:14:26.595936+0000","last_unstale":"2024-05-09T07:39:34.231279+0000","last_undegraded":"2024-05-09T07:39:34.231279+0000","last_fullsized":"2024-05-09T07:39:34.231279+0000","mapping_epoch":26,"log_start":"27'2010","ondisk_log_start":"27'2010","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7157,"log_dups_size":0,"ondisk_log_size":7157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:45.938473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040384,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":305,"num_read_kb":77470,"num_write":6442,"num_write_kb":989255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2602","reported_seq":2637,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.576974+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:39:34.576974+0000","last_peered":"2024-05-09T07:39:34.576974+0000","last_clean":"2024-05-09T07:39:34.576974+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:39:34.576974+0000","last_undegraded":"2024-05-09T07:39:34.576974+0000","last_fullsized":"2024-05-09T07:39:34.576974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2602,"log_dups_size":0,"ondisk_log_size":2602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5738","reported_seq":5776,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:06.691749+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:39:06.691749+0000","last_peered":"2024-05-09T07:39:06.691749+0000","last_clean":"2024-05-09T07:39:06.691749+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:39:06.691749+0000","last_undegraded":"2024-05-09T07:39:06.691749+0000","last_fullsized":"2024-05-09T07:39:06.691749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5739,"num_write_kb":52939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9112","reported_seq":9479,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.730627+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:39:33.730627+0000","last_peered":"2024-05-09T07:39:33.730627+0000","last_clean":"2024-05-09T07:39:33.730627+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:39:33.730627+0000","last_undegraded":"2024-05-09T07:39:33.730627+0000","last_fullsized":"2024-05-09T07:39:33.730627+0000","mapping_epoch":22,"log_start":"27'2805","ondisk_log_start":"27'2805","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6307,"log_dups_size":0,"ondisk_log_size":6307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1304576,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":335,"num_read_kb":85090,"num_write":6440,"num_write_kb":967374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2569","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.769929+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:39:33.769929+0000","last_peered":"2024-05-09T07:39:33.769929+0000","last_clean":"2024-05-09T07:39:33.769929+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:39:33.769929+0000","last_undegraded":"2024-05-09T07:39:33.769929+0000","last_fullsized":"2024-05-09T07:39:33.769929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'5942","reported_seq":6846,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:15.634681+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:38:15.634681+0000","last_peered":"2024-05-09T07:38:15.634681+0000","last_clean":"2024-05-09T07:38:15.634681+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:38:15.634681+0000","last_undegraded":"2024-05-09T07:38:15.634681+0000","last_fullsized":"2024-05-09T07:38:15.634681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":867,"num_read_kb":4659,"num_write":5943,"num_write_kb":49048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9593","reported_seq":9981,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.605018+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:39:33.605018+0000","last_peered":"2024-05-09T07:39:33.605018+0000","last_clean":"2024-05-09T07:39:33.605018+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:39:33.605018+0000","last_undegraded":"2024-05-09T07:39:33.605018+0000","last_fullsized":"2024-05-09T07:39:33.605018+0000","mapping_epoch":22,"log_start":"27'3323","ondisk_log_start":"27'3323","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746850,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6812,"num_write_kb":1059940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9162","reported_seq":9389,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.603712+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:39:34.603712+0000","last_peered":"2024-05-09T07:39:34.603712+0000","last_clean":"2024-05-09T07:39:34.603712+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:39:34.603712+0000","last_undegraded":"2024-05-09T07:39:34.603712+0000","last_fullsized":"2024-05-09T07:39:34.603712+0000","mapping_epoch":22,"log_start":"27'1813","ondisk_log_start":"27'1813","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7349,"log_dups_size":0,"ondisk_log_size":7349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":49276,"num_write":6559,"num_write_kb":912347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2528","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.594958+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:39:34.594958+0000","last_peered":"2024-05-09T07:39:34.594958+0000","last_clean":"2024-05-09T07:39:34.594958+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:39:34.594958+0000","last_undegraded":"2024-05-09T07:39:34.594958+0000","last_fullsized":"2024-05-09T07:39:34.594958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9067","reported_seq":9105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:08.929933+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:38:08.929933+0000","last_peered":"2024-05-09T07:38:08.929933+0000","last_clean":"2024-05-09T07:38:08.929933+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:38:08.929933+0000","last_undegraded":"2024-05-09T07:38:08.929933+0000","last_fullsized":"2024-05-09T07:38:08.929933+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9064,"num_write_kb":67566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8837","reported_seq":10023,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.965223+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:39:33.965223+0000","last_peered":"2024-05-09T07:39:33.965223+0000","last_clean":"2024-05-09T07:39:33.965223+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:39:33.965223+0000","last_undegraded":"2024-05-09T07:39:33.965223+0000","last_fullsized":"2024-05-09T07:39:33.965223+0000","mapping_epoch":22,"log_start":"27'900","ondisk_log_start":"27'900","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7937,"log_dups_size":0,"ondisk_log_size":7937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1948104,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1153,"num_read_kb":292862,"num_write":6075,"num_write_kb":877058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2618","reported_seq":2654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.701610+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:39:33.701610+0000","last_peered":"2024-05-09T07:39:33.701610+0000","last_clean":"2024-05-09T07:39:33.701610+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:39:33.701610+0000","last_undegraded":"2024-05-09T07:39:33.701610+0000","last_fullsized":"2024-05-09T07:39:33.701610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2618,"log_dups_size":0,"ondisk_log_size":2618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5353","reported_seq":5391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:13.460729+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:38:13.460729+0000","last_peered":"2024-05-09T07:38:13.460729+0000","last_clean":"2024-05-09T07:38:13.460729+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:38:13.460729+0000","last_undegraded":"2024-05-09T07:38:13.460729+0000","last_fullsized":"2024-05-09T07:38:13.460729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5353,"log_dups_size":0,"ondisk_log_size":5353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5353,"num_write_kb":44481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9693","reported_seq":9731,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:43.717707+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:38:43.717707+0000","last_peered":"2024-05-09T07:38:43.717707+0000","last_clean":"2024-05-09T07:38:43.717707+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:38:43.717707+0000","last_undegraded":"2024-05-09T07:38:43.717707+0000","last_fullsized":"2024-05-09T07:38:43.717707+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7993,"log_dups_size":0,"ondisk_log_size":7993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9690,"num_write_kb":72019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2555","reported_seq":2590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.417888+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:39:34.417888+0000","last_peered":"2024-05-09T07:39:34.417888+0000","last_clean":"2024-05-09T07:39:34.417888+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:39:34.417888+0000","last_undegraded":"2024-05-09T07:39:34.417888+0000","last_fullsized":"2024-05-09T07:39:34.417888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9392","reported_seq":9447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.677629+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:39:33.677629+0000","last_peered":"2024-05-09T07:39:33.677629+0000","last_clean":"2024-05-09T07:39:33.677629+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:39:33.677629+0000","last_undegraded":"2024-05-09T07:39:33.677629+0000","last_fullsized":"2024-05-09T07:39:33.677629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1565758,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":24,"num_read_kb":6096,"num_write":6594,"num_write_kb":983207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8024","reported_seq":8062,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:48.275994+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:38:48.275994+0000","last_peered":"2024-05-09T07:38:48.275994+0000","last_clean":"2024-05-09T07:38:48.275994+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:38:48.275994+0000","last_undegraded":"2024-05-09T07:38:48.275994+0000","last_fullsized":"2024-05-09T07:38:48.275994+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6324,"log_dups_size":0,"ondisk_log_size":6324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8026,"num_write_kb":66635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2597","reported_seq":2632,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.242028+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:39:34.242028+0000","last_peered":"2024-05-09T07:39:34.242028+0000","last_clean":"2024-05-09T07:39:34.242028+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:39:34.242028+0000","last_undegraded":"2024-05-09T07:39:34.242028+0000","last_fullsized":"2024-05-09T07:39:34.242028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9823","reported_seq":9975,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.979566+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:39:33.979566+0000","last_peered":"2024-05-09T07:39:33.979566+0000","last_clean":"2024-05-09T07:39:33.979566+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:39:33.979566+0000","last_undegraded":"2024-05-09T07:39:33.979566+0000","last_fullsized":"2024-05-09T07:39:33.979566+0000","mapping_epoch":22,"log_start":"27'3543","ondisk_log_start":"27'3543","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081890,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":121,"num_read_kb":30734,"num_write":7010,"num_write_kb":1030226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2491","reported_seq":2529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.847333+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:39:33.847333+0000","last_peered":"2024-05-09T07:39:33.847333+0000","last_clean":"2024-05-09T07:39:33.847333+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:39:33.847333+0000","last_undegraded":"2024-05-09T07:39:33.847333+0000","last_fullsized":"2024-05-09T07:39:33.847333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2491,"log_dups_size":0,"ondisk_log_size":2491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2611","reported_seq":2648,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.123638+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:39:33.123638+0000","last_peered":"2024-05-09T07:39:33.123638+0000","last_clean":"2024-05-09T07:39:33.123638+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:39:33.123638+0000","last_undegraded":"2024-05-09T07:39:33.123638+0000","last_fullsized":"2024-05-09T07:39:33.123638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2515","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.622088+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:39:33.622088+0000","last_peered":"2024-05-09T07:39:33.622088+0000","last_clean":"2024-05-09T07:39:33.622088+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:39:33.622088+0000","last_undegraded":"2024-05-09T07:39:33.622088+0000","last_fullsized":"2024-05-09T07:39:33.622088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2604","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.371669+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:39:34.371669+0000","last_peered":"2024-05-09T07:39:34.371669+0000","last_clean":"2024-05-09T07:39:34.371669+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:39:34.371669+0000","last_undegraded":"2024-05-09T07:39:34.371669+0000","last_fullsized":"2024-05-09T07:39:34.371669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2604,"log_dups_size":0,"ondisk_log_size":2604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2596","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.755716+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:39:33.755716+0000","last_peered":"2024-05-09T07:39:33.755716+0000","last_clean":"2024-05-09T07:39:33.755716+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:39:33.755716+0000","last_undegraded":"2024-05-09T07:39:33.755716+0000","last_fullsized":"2024-05-09T07:39:33.755716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2482","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.054999+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:39:34.054999+0000","last_peered":"2024-05-09T07:39:34.054999+0000","last_clean":"2024-05-09T07:39:34.054999+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:39:34.054999+0000","last_undegraded":"2024-05-09T07:39:34.054999+0000","last_fullsized":"2024-05-09T07:39:34.054999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2610","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.002968+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:39:34.002968+0000","last_peered":"2024-05-09T07:39:34.002968+0000","last_clean":"2024-05-09T07:39:34.002968+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:39:34.002968+0000","last_undegraded":"2024-05-09T07:39:34.002968+0000","last_fullsized":"2024-05-09T07:39:34.002968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2610,"log_dups_size":0,"ondisk_log_size":2610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2582","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.942776+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:39:33.942776+0000","last_peered":"2024-05-09T07:39:33.942776+0000","last_clean":"2024-05-09T07:39:33.942776+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:39:33.942776+0000","last_undegraded":"2024-05-09T07:39:33.942776+0000","last_fullsized":"2024-05-09T07:39:33.942776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2566","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.702866+0000","last_change":"2024-05-09T07:14:26.646404+0000","last_active":"2024-05-09T07:39:33.702866+0000","last_peered":"2024-05-09T07:39:33.702866+0000","last_clean":"2024-05-09T07:39:33.702866+0000","last_became_active":"2024-05-09T07:14:26.646058+0000","last_became_peered":"2024-05-09T07:14:26.646058+0000","last_unstale":"2024-05-09T07:39:33.702866+0000","last_undegraded":"2024-05-09T07:39:33.702866+0000","last_fullsized":"2024-05-09T07:39:33.702866+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2566,"log_dups_size":0,"ondisk_log_size":2566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'2525","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.518809+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:39:33.518809+0000","last_peered":"2024-05-09T07:39:33.518809+0000","last_clean":"2024-05-09T07:39:33.518809+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:39:33.518809+0000","last_undegraded":"2024-05-09T07:39:33.518809+0000","last_fullsized":"2024-05-09T07:39:33.518809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2599","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.840412+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:39:33.840412+0000","last_peered":"2024-05-09T07:39:33.840412+0000","last_clean":"2024-05-09T07:39:33.840412+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:39:33.840412+0000","last_undegraded":"2024-05-09T07:39:33.840412+0000","last_fullsized":"2024-05-09T07:39:33.840412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2608","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.225626+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:39:34.225626+0000","last_peered":"2024-05-09T07:39:34.225626+0000","last_clean":"2024-05-09T07:39:34.225626+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:39:34.225626+0000","last_undegraded":"2024-05-09T07:39:34.225626+0000","last_fullsized":"2024-05-09T07:39:34.225626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2601","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:32.409252+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:39:32.409252+0000","last_peered":"2024-05-09T07:39:32.409252+0000","last_clean":"2024-05-09T07:39:32.409252+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:39:32.409252+0000","last_undegraded":"2024-05-09T07:39:32.409252+0000","last_fullsized":"2024-05-09T07:39:32.409252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2606","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.135665+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:39:34.135665+0000","last_peered":"2024-05-09T07:39:34.135665+0000","last_clean":"2024-05-09T07:39:34.135665+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:39:34.135665+0000","last_undegraded":"2024-05-09T07:39:34.135665+0000","last_fullsized":"2024-05-09T07:39:34.135665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2544","reported_seq":2580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.079417+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:39:33.079417+0000","last_peered":"2024-05-09T07:39:33.079417+0000","last_clean":"2024-05-09T07:39:33.079417+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:39:33.079417+0000","last_undegraded":"2024-05-09T07:39:33.079417+0000","last_fullsized":"2024-05-09T07:39:33.079417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2609","reported_seq":2646,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.792677+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:39:33.792677+0000","last_peered":"2024-05-09T07:39:33.792677+0000","last_clean":"2024-05-09T07:39:33.792677+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:39:33.792677+0000","last_undegraded":"2024-05-09T07:39:33.792677+0000","last_fullsized":"2024-05-09T07:39:33.792677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2609,"log_dups_size":0,"ondisk_log_size":2609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2547","reported_seq":2584,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.736964+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:39:33.736964+0000","last_peered":"2024-05-09T07:39:33.736964+0000","last_clean":"2024-05-09T07:39:33.736964+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:39:33.736964+0000","last_undegraded":"2024-05-09T07:39:33.736964+0000","last_fullsized":"2024-05-09T07:39:33.736964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2547,"log_dups_size":0,"ondisk_log_size":2547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2669","reported_seq":2706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.483917+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:39:34.483917+0000","last_peered":"2024-05-09T07:39:34.483917+0000","last_clean":"2024-05-09T07:39:34.483917+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:39:34.483917+0000","last_undegraded":"2024-05-09T07:39:34.483917+0000","last_fullsized":"2024-05-09T07:39:34.483917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2669,"log_dups_size":0,"ondisk_log_size":2669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2520","reported_seq":2556,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.617853+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:39:34.617853+0000","last_peered":"2024-05-09T07:39:34.617853+0000","last_clean":"2024-05-09T07:39:34.617853+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:39:34.617853+0000","last_undegraded":"2024-05-09T07:39:34.617853+0000","last_fullsized":"2024-05-09T07:39:34.617853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2504","reported_seq":2540,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.803579+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:39:33.803579+0000","last_peered":"2024-05-09T07:39:33.803579+0000","last_clean":"2024-05-09T07:39:33.803579+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:39:33.803579+0000","last_undegraded":"2024-05-09T07:39:33.803579+0000","last_fullsized":"2024-05-09T07:39:33.803579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2504,"log_dups_size":0,"ondisk_log_size":2504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2482","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.172679+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:39:34.172679+0000","last_peered":"2024-05-09T07:39:34.172679+0000","last_clean":"2024-05-09T07:39:34.172679+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:39:34.172679+0000","last_undegraded":"2024-05-09T07:39:34.172679+0000","last_fullsized":"2024-05-09T07:39:34.172679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2503","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.838109+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:39:33.838109+0000","last_peered":"2024-05-09T07:39:33.838109+0000","last_clean":"2024-05-09T07:39:33.838109+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:39:33.838109+0000","last_undegraded":"2024-05-09T07:39:33.838109+0000","last_fullsized":"2024-05-09T07:39:33.838109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2632","reported_seq":2668,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.753412+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:39:33.753412+0000","last_peered":"2024-05-09T07:39:33.753412+0000","last_clean":"2024-05-09T07:39:33.753412+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:39:33.753412+0000","last_undegraded":"2024-05-09T07:39:33.753412+0000","last_fullsized":"2024-05-09T07:39:33.753412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2632,"log_dups_size":0,"ondisk_log_size":2632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2599","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.441391+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:39:34.441391+0000","last_peered":"2024-05-09T07:39:34.441391+0000","last_clean":"2024-05-09T07:39:34.441391+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:39:34.441391+0000","last_undegraded":"2024-05-09T07:39:34.441391+0000","last_fullsized":"2024-05-09T07:39:34.441391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2537","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.832419+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:39:33.832419+0000","last_peered":"2024-05-09T07:39:33.832419+0000","last_clean":"2024-05-09T07:39:33.832419+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:39:33.832419+0000","last_undegraded":"2024-05-09T07:39:33.832419+0000","last_fullsized":"2024-05-09T07:39:33.832419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2537,"log_dups_size":0,"ondisk_log_size":2537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.487057+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:39:34.487057+0000","last_peered":"2024-05-09T07:39:34.487057+0000","last_clean":"2024-05-09T07:39:34.487057+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:39:34.487057+0000","last_undegraded":"2024-05-09T07:39:34.487057+0000","last_fullsized":"2024-05-09T07:39:34.487057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2503","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.726849+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:39:33.726849+0000","last_peered":"2024-05-09T07:39:33.726849+0000","last_clean":"2024-05-09T07:39:33.726849+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:39:33.726849+0000","last_undegraded":"2024-05-09T07:39:33.726849+0000","last_fullsized":"2024-05-09T07:39:33.726849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2542","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.644003+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:39:34.644003+0000","last_peered":"2024-05-09T07:39:34.644003+0000","last_clean":"2024-05-09T07:39:34.644003+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:39:34.644003+0000","last_undegraded":"2024-05-09T07:39:34.644003+0000","last_fullsized":"2024-05-09T07:39:34.644003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2580","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.459093+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:39:33.459093+0000","last_peered":"2024-05-09T07:39:33.459093+0000","last_clean":"2024-05-09T07:39:33.459093+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:39:33.459093+0000","last_undegraded":"2024-05-09T07:39:33.459093+0000","last_fullsized":"2024-05-09T07:39:33.459093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2658","reported_seq":2695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.225175+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:39:34.225175+0000","last_peered":"2024-05-09T07:39:34.225175+0000","last_clean":"2024-05-09T07:39:34.225175+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:39:34.225175+0000","last_undegraded":"2024-05-09T07:39:34.225175+0000","last_fullsized":"2024-05-09T07:39:34.225175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2496","reported_seq":2534,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.141290+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:39:34.141290+0000","last_peered":"2024-05-09T07:39:34.141290+0000","last_clean":"2024-05-09T07:39:34.141290+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:39:34.141290+0000","last_undegraded":"2024-05-09T07:39:34.141290+0000","last_fullsized":"2024-05-09T07:39:34.141290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2482","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.536252+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:39:34.536252+0000","last_peered":"2024-05-09T07:39:34.536252+0000","last_clean":"2024-05-09T07:39:34.536252+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:39:34.536252+0000","last_undegraded":"2024-05-09T07:39:34.536252+0000","last_fullsized":"2024-05-09T07:39:34.536252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":61502438,"num_objects":168,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":12270,"num_read_kb":3117350,"num_write":214562,"num_write_kb":30722105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120573952,"data_stored":120573952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232059,"ondisk_log_size":232059,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164175,"ondisk_log_size":164175,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":248505113,"num_objects":195,"num_object_clones":0,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":195,"num_whiteouts":0,"num_read":3177,"num_read_kb":19424,"num_write":211653,"num_write_kb":1695367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":127,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497180672,"data_stored":497116653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192901,"ondisk_log_size":192901,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575163,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":970376,"kb_used_data":64640,"kb_used_omap":1,"kb_used_meta":905726,"kb_avail":93401464,"statfs":{"total":96636764160,"available":95643099136,"internally_reserved":0,"allocated":66191360,"data_stored":66012455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":927463882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575162,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1129536,"kb_used_data":91960,"kb_used_omap":1,"kb_used_meta":1037566,"kb_avail":93242304,"statfs":{"total":96636764160,"available":95480119296,"internally_reserved":0,"allocated":94167040,"data_stored":93987681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1062468038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575162,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1019696,"kb_used_data":81896,"kb_used_omap":1,"kb_used_meta":937662,"kb_avail":93352144,"statfs":{"total":96636764160,"available":95592595456,"internally_reserved":0,"allocated":83861504,"data_stored":83687882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960166341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575161,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062008,"kb_used_data":84016,"kb_used_omap":1,"kb_used_meta":977854,"kb_avail":93309832,"statfs":{"total":96636764160,"available":95549267968,"internally_reserved":0,"allocated":86032384,"data_stored":85850833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001322950},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575163,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279116,"kb_used_data":128116,"kb_used_omap":1,"kb_used_meta":1150078,"kb_avail":93092724,"statfs":{"total":96636764160,"available":95326949376,"internally_reserved":0,"allocated":131190784,"data_stored":131013875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1177680327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729932,"kb_used_data":76932,"kb_used_omap":1,"kb_used_meta":652990,"kb_avail":93641908,"statfs":{"total":96636764160,"available":95889313792,"internally_reserved":0,"allocated":78778368,"data_stored":78606656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668662213},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575165,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1251056,"kb_used_data":82072,"kb_used_omap":1,"kb_used_meta":1168062,"kb_avail":93120784,"statfs":{"total":96636764160,"available":95355682816,"internally_reserved":0,"allocated":84041728,"data_stored":83864307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1196095941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898152,"kb_used_data":68896,"kb_used_omap":1,"kb_used_meta":829246,"kb_avail":93473688,"statfs":{"total":96636764160,"available":95717056512,"internally_reserved":0,"allocated":70549504,"data_stored":70364913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":849148357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67109502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54583296,"data_stored":54582378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71409664,"data_stored":71398347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46141440,"data_stored":46137434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":68136960,"data_stored":68126990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":55549952,"data_stored":55543988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2760704,"data_stored":2760704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23805952,"data_stored":23805952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29478912,"data_stored":29478912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16297984,"data_stored":16297984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21856256,"data_stored":21856256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24752128,"data_stored":24752128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":573440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:39:38.426 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:39:38.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.552+0000 7fc1150ae640 1 Processor -- start 2024-05-09T07:39:38.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.552+0000 7fc1150ae640 1 -- start start 2024-05-09T07:39:38.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc1150ae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100be320 0x7fc1100c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc1150ae640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc1100c8b90 0x7fc110155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.564 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc1150ae640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc110159840 con 0x7fc1100c5610 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc1150ae640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc110159310 con 0x7fc1100c8b90 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc1150ae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc110159560 con 0x7fc1100be320 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc10f59e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc1100c8b90 0x7fc110155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc10ed9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc1100c5610 0x7fc110155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34598/0 (socket says 172.21.15.160:34598) 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.553+0000 7fc10ed9d640 1 -- 172.21.15.160:0/3159522071 learned_addr learned my addr 172.21.15.160:0/3159522071 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10f59e640 1 -- 172.21.15.160:0/3159522071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100be320 msgr2=0x7fc1100c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10f59e640 1 --2- 172.21.15.160:0/3159522071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100be320 0x7fc1100c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10f59e640 1 -- 172.21.15.160:0/3159522071 >> v1:172.21.15.160:6789/0 conn(0x7fc1100c5610 legacy=0x7fc110155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10dd9b640 1 -- 172.21.15.160:0/3159522071 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866306620 0 0) 0x7fc110159840 con 0x7fc1100c5610 2024-05-09T07:39:38.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10f59e640 1 -- 172.21.15.160:0/3159522071 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1100c77d0 con 0x7fc1100c8b90 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10f59e640 1 --2- 172.21.15.160:0/3159522071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c8b90 0x7fc110155a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc108009d90 tx=0x7fc10802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c1b4c041812f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.554+0000 7fc10dd9b640 1 -- 172.21.15.160:0/3159522071 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108007740 con 0x7fc1100c8b90 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc10dd9b640 1 -- 172.21.15.160:0/3159522071 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1080078e0 con 0x7fc1100c8b90 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc10dd9b640 1 -- 172.21.15.160:0/3159522071 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc108007cb0 con 0x7fc1100c8b90 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 -- 172.21.15.160:0/3159522071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c8b90 msgr2=0x7fc110155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 --2- 172.21.15.160:0/3159522071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c8b90 0x7fc110155a20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc108009d90 tx=0x7fc10802a530 comp rx=0 tx=0).stop 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 -- 172.21.15.160:0/3159522071 shutdown_connections 2024-05-09T07:39:38.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 --2- 172.21.15.160:0/3159522071 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c8b90 0x7fc110155a20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 --2- 172.21.15.160:0/3159522071 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100be320 0x7fc1100c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 -- 172.21.15.160:0/3159522071 >> 172.21.15.160:0/3159522071 conn(0x7fc110072e90 msgr2=0x7fc1100ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:38.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 -- 172.21.15.160:0/3159522071 shutdown_connections 2024-05-09T07:39:38.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.555+0000 7fc1150ae640 1 -- 172.21.15.160:0/3159522071 wait complete. 2024-05-09T07:39:38.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.556+0000 7fc1150ae640 1 Processor -- start 2024-05-09T07:39:38.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.556+0000 7fc1150ae640 1 -- start start 2024-05-09T07:39:38.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc1150ae640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1100be320 0x7fc11014bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc1150ae640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 0x7fc110156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc1150ae640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100c8b90 0x7fc110157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc1150ae640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100ad6a0 con 0x7fc1100c5610 2024-05-09T07:39:38.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc1150ae640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc1100c7d90 con 0x7fc1100be320 2024-05-09T07:39:38.571 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc1150ae640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc110155f60 con 0x7fc1100c8b90 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 0x7fc110156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 0x7fc110156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34704/0 (socket says 172.21.15.160:34704) 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 -- 172.21.15.160:0/1527169833 learned_addr learned my addr 172.21.15.160:0/1527169833 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10f59e640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100c8b90 0x7fc110157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10ed9d640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1100be320 0x7fc11014bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 -- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100c8b90 msgr2=0x7fc110157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100c8b90 0x7fc110157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 -- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1100be320 msgr2=0x7fc11014bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1100be320 0x7fc11014bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10e59c640 1 -- 172.21.15.160:0/1527169833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc110158970 con 0x7fc1100c5610 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.557+0000 7fc10ed9d640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1100be320 0x7fc11014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.558+0000 7fc10e59c640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 0x7fc110156c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800b450 tx=0x7fc0f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.558+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8015020 con 0x7fc1100c5610 2024-05-09T07:39:38.572 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.558+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc0f8011890 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.558+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc110158c50 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.558+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8011c80 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.558+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1100c2140 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.559+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1100756c0 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.559+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc0f8018020 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.560+0000 7fc1077fe640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc0f006be30 0x7fc0f006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.560+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc0f808b390 con 0x7fc1100c5610 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.560+0000 7fc10ed9d640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc0f006be30 0x7fc0f006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.573 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.560+0000 7fc10ed9d640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc0f006be30 0x7fc0f006e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc100000f60 tx=0x7fc100009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.574 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.564+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc0f801f020 con 0x7fc1100c5610 2024-05-09T07:39:38.757 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.746+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1100c27c0 con 0x7fc0f006be30 2024-05-09T07:39:38.764 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.754+0000 7fc1077fe640 1 -- 172.21.15.160:0/1527169833 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336227 (secure 0 0 0) 0x7fc1100c27c0 con 0x7fc0f006be30 2024-05-09T07:39:38.768 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:39:38.770 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc0f006be30 msgr2=0x7fc0f006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc0f006be30 0x7fc0f006e2f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc100000f60 tx=0x7fc100009040 comp rx=0 tx=0).stop 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 msgr2=0x7fc110156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 0x7fc110156c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800b450 tx=0x7fc0f800b910 comp rx=0 tx=0).stop 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 shutdown_connections 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc0f006be30 0x7fc0f006e2f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc1100c8b90 0x7fc110157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc1100c5610 0x7fc110156c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 --2- 172.21.15.160:0/1527169833 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc1100be320 0x7fc11014bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 >> 172.21.15.160:0/1527169833 conn(0x7fc110072e90 msgr2=0x7fc1100aa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 shutdown_connections 2024-05-09T07:39:38.772 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.757+0000 7fc1150ae640 1 -- 172.21.15.160:0/1527169833 wait complete. 2024-05-09T07:39:38.779 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":802,"stamp":"2024-05-09T07:39:37.561520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310597919,"num_objects":478,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":478,"num_whiteouts":0,"num_read":15563,"num_read_kb":3136874,"num_write":427381,"num_write_kb":32419888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":127,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589252,"ondisk_log_size":589252,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8339872,"kb_used_data":678528,"kb_used_omap":12,"kb_used_meta":7659187,"kb_avail":746634848,"statfs":{"total":773094113280,"available":764554084352,"internally_reserved":0,"allocated":694812672,"data_stored":693388602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":7843008049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,6,4,12,18,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-148056247,"num_objects":-1063,"num_object_clones":0,"num_object_copies":-3364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1063,"num_whiteouts":0,"num_read":84,"num_read_kb":17248,"num_write":3915,"num_write_kb":86342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003252"},"pg_stats":[{"pgid":"2.1f","version":"27'2672","reported_seq":2711,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:51.735149+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:38:51.735149+0000","last_peered":"2024-05-09T07:38:51.735149+0000","last_clean":"2024-05-09T07:38:51.735149+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:38:51.735149+0000","last_undegraded":"2024-05-09T07:38:51.735149+0000","last_fullsized":"2024-05-09T07:38:51.735149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":2684,"num_write_kb":18241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2569","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.929399+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:39:33.929399+0000","last_peered":"2024-05-09T07:39:33.929399+0000","last_clean":"2024-05-09T07:39:33.929399+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:39:33.929399+0000","last_undegraded":"2024-05-09T07:39:33.929399+0000","last_fullsized":"2024-05-09T07:39:33.929399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9807","reported_seq":10237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.981916+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:39:33.981916+0000","last_peered":"2024-05-09T07:39:33.981916+0000","last_clean":"2024-05-09T07:39:33.981916+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:39:33.981916+0000","last_undegraded":"2024-05-09T07:39:33.981916+0000","last_fullsized":"2024-05-09T07:39:33.981916+0000","mapping_epoch":22,"log_start":"27'3504","ondisk_log_start":"27'3504","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6303,"log_dups_size":0,"ondisk_log_size":6303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6872883,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":399,"num_read_kb":101346,"num_write":7160,"num_write_kb":1009567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'7784","reported_seq":7824,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:23.622641+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:39:23.622641+0000","last_peered":"2024-05-09T07:39:23.622641+0000","last_clean":"2024-05-09T07:39:23.622641+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:39:23.622641+0000","last_undegraded":"2024-05-09T07:39:23.622641+0000","last_fullsized":"2024-05-09T07:39:23.622641+0000","mapping_epoch":17,"log_start":"27'1500","ondisk_log_start":"27'1500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":7796,"num_write_kb":54453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2550","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.085746+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:39:34.085746+0000","last_peered":"2024-05-09T07:39:34.085746+0000","last_clean":"2024-05-09T07:39:34.085746+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:39:34.085746+0000","last_undegraded":"2024-05-09T07:39:34.085746+0000","last_fullsized":"2024-05-09T07:39:34.085746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9531","reported_seq":9918,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.127850+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:39:34.127850+0000","last_peered":"2024-05-09T07:39:34.127850+0000","last_clean":"2024-05-09T07:39:34.127850+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:39:34.127850+0000","last_undegraded":"2024-05-09T07:39:34.127850+0000","last_fullsized":"2024-05-09T07:39:34.127850+0000","mapping_epoch":22,"log_start":"27'2329","ondisk_log_start":"27'2329","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6759,"num_write_kb":1095702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9522","reported_seq":9695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.191922+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:39:33.191922+0000","last_peered":"2024-05-09T07:39:33.191922+0000","last_clean":"2024-05-09T07:39:33.191922+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:39:33.191922+0000","last_undegraded":"2024-05-09T07:39:33.191922+0000","last_fullsized":"2024-05-09T07:39:33.191922+0000","mapping_epoch":17,"log_start":"27'2500","ondisk_log_start":"27'2500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7022,"log_dups_size":0,"ondisk_log_size":7022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5211758,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":1083,"num_write":9520,"num_write_kb":77042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2657","reported_seq":2693,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.656593+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:39:33.656593+0000","last_peered":"2024-05-09T07:39:33.656593+0000","last_clean":"2024-05-09T07:39:33.656593+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:39:33.656593+0000","last_undegraded":"2024-05-09T07:39:33.656593+0000","last_fullsized":"2024-05-09T07:39:33.656593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2657,"log_dups_size":0,"ondisk_log_size":2657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9063","reported_seq":9472,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.136390+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:39:34.136390+0000","last_peered":"2024-05-09T07:39:34.136390+0000","last_clean":"2024-05-09T07:39:34.136390+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:39:34.136390+0000","last_undegraded":"2024-05-09T07:39:34.136390+0000","last_fullsized":"2024-05-09T07:39:34.136390+0000","mapping_epoch":22,"log_start":"27'1901","ondisk_log_start":"27'1901","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917224,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6298,"num_write_kb":900735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4288","reported_seq":4325,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.588906+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:39:33.588906+0000","last_peered":"2024-05-09T07:39:33.588906+0000","last_clean":"2024-05-09T07:39:33.588906+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:39:33.588906+0000","last_undegraded":"2024-05-09T07:39:33.588906+0000","last_fullsized":"2024-05-09T07:39:33.588906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":4301,"num_write_kb":47774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2587","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.753943+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:39:33.753943+0000","last_peered":"2024-05-09T07:39:33.753943+0000","last_clean":"2024-05-09T07:39:33.753943+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:39:33.753943+0000","last_undegraded":"2024-05-09T07:39:33.753943+0000","last_fullsized":"2024-05-09T07:39:33.753943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'9248","reported_seq":9721,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.271401+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:39:34.271401+0000","last_peered":"2024-05-09T07:39:34.271401+0000","last_clean":"2024-05-09T07:39:34.271401+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:39:34.271401+0000","last_undegraded":"2024-05-09T07:39:34.271401+0000","last_fullsized":"2024-05-09T07:39:34.271401+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":442,"num_read_kb":112268,"num_write":6537,"num_write_kb":853848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2578","reported_seq":2615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.930484+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:39:33.930484+0000","last_peered":"2024-05-09T07:39:33.930484+0000","last_clean":"2024-05-09T07:39:33.930484+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:39:33.930484+0000","last_undegraded":"2024-05-09T07:39:33.930484+0000","last_fullsized":"2024-05-09T07:39:33.930484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6961","reported_seq":6999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:03.399820+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:39:03.399820+0000","last_peered":"2024-05-09T07:39:03.399820+0000","last_clean":"2024-05-09T07:39:03.399820+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:39:03.399820+0000","last_undegraded":"2024-05-09T07:39:03.399820+0000","last_fullsized":"2024-05-09T07:39:03.399820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6961,"log_dups_size":0,"ondisk_log_size":6961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6969,"num_write_kb":53856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9806","reported_seq":10053,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.172117+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:39:34.172117+0000","last_peered":"2024-05-09T07:39:34.172117+0000","last_clean":"2024-05-09T07:39:34.172117+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:39:34.172117+0000","last_undegraded":"2024-05-09T07:39:34.172117+0000","last_fullsized":"2024-05-09T07:39:34.172117+0000","mapping_epoch":22,"log_start":"27'2633","ondisk_log_start":"27'2633","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":54610,"num_write":7039,"num_write_kb":1008697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2599","reported_seq":2636,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.034015+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:39:34.034015+0000","last_peered":"2024-05-09T07:39:34.034015+0000","last_clean":"2024-05-09T07:39:34.034015+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:39:34.034015+0000","last_undegraded":"2024-05-09T07:39:34.034015+0000","last_fullsized":"2024-05-09T07:39:34.034015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7778","reported_seq":7817,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:42.793042+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:38:42.793042+0000","last_peered":"2024-05-09T07:38:42.793042+0000","last_clean":"2024-05-09T07:38:42.793042+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:38:42.793042+0000","last_undegraded":"2024-05-09T07:38:42.793042+0000","last_fullsized":"2024-05-09T07:38:42.793042+0000","mapping_epoch":17,"log_start":"27'400","ondisk_log_start":"27'400","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":58435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9627","reported_seq":10167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.260189+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:39:34.260189+0000","last_peered":"2024-05-09T07:39:34.260189+0000","last_clean":"2024-05-09T07:39:34.260189+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:39:34.260189+0000","last_undegraded":"2024-05-09T07:39:34.260189+0000","last_fullsized":"2024-05-09T07:39:34.260189+0000","mapping_epoch":22,"log_start":"27'2443","ondisk_log_start":"27'2443","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2628189,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":508,"num_read_kb":129802,"num_write":6847,"num_write_kb":970185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2672","reported_seq":2709,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.851191+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:39:33.851191+0000","last_peered":"2024-05-09T07:39:33.851191+0000","last_clean":"2024-05-09T07:39:33.851191+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:39:33.851191+0000","last_undegraded":"2024-05-09T07:39:33.851191+0000","last_fullsized":"2024-05-09T07:39:33.851191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3774","reported_seq":3813,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:40.266938+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:37:40.266938+0000","last_peered":"2024-05-09T07:37:40.266938+0000","last_clean":"2024-05-09T07:37:40.266938+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:37:40.266938+0000","last_undegraded":"2024-05-09T07:37:40.266938+0000","last_fullsized":"2024-05-09T07:37:40.266938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3774,"log_dups_size":0,"ondisk_log_size":3774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3771,"num_write_kb":27117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9040","reported_seq":9652,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.430298+0000","last_change":"2024-05-09T07:14:26.646797+0000","last_active":"2024-05-09T07:39:33.430298+0000","last_peered":"2024-05-09T07:39:33.430298+0000","last_clean":"2024-05-09T07:39:33.430298+0000","last_became_active":"2024-05-09T07:14:26.646222+0000","last_became_peered":"2024-05-09T07:14:26.646222+0000","last_unstale":"2024-05-09T07:39:33.430298+0000","last_undegraded":"2024-05-09T07:39:33.430298+0000","last_fullsized":"2024-05-09T07:39:33.430298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3553201,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6364,"num_write_kb":893211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2580","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.213138+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:39:34.213138+0000","last_peered":"2024-05-09T07:39:34.213138+0000","last_clean":"2024-05-09T07:39:34.213138+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:39:34.213138+0000","last_undegraded":"2024-05-09T07:39:34.213138+0000","last_fullsized":"2024-05-09T07:39:34.213138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2263","reported_seq":2301,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:10.954485+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:37:10.954485+0000","last_peered":"2024-05-09T07:37:10.954485+0000","last_clean":"2024-05-09T07:37:10.954485+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:37:10.954485+0000","last_undegraded":"2024-05-09T07:37:10.954485+0000","last_fullsized":"2024-05-09T07:37:10.954485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2272,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9928","reported_seq":10064,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.216405+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:39:34.216405+0000","last_peered":"2024-05-09T07:39:34.216405+0000","last_clean":"2024-05-09T07:39:34.216405+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:39:34.216405+0000","last_undegraded":"2024-05-09T07:39:34.216405+0000","last_fullsized":"2024-05-09T07:39:34.216405+0000","mapping_epoch":22,"log_start":"27'2728","ondisk_log_start":"27'2728","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1371263,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":26416,"num_write":7233,"num_write_kb":974347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.954387+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:39:33.954387+0000","last_peered":"2024-05-09T07:39:33.954387+0000","last_clean":"2024-05-09T07:39:33.954387+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:39:33.954387+0000","last_undegraded":"2024-05-09T07:39:33.954387+0000","last_fullsized":"2024-05-09T07:39:33.954387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4066","reported_seq":4105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:39.865043+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:38:39.865043+0000","last_peered":"2024-05-09T07:38:39.865043+0000","last_clean":"2024-05-09T07:38:39.865043+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:38:39.865043+0000","last_undegraded":"2024-05-09T07:38:39.865043+0000","last_fullsized":"2024-05-09T07:38:39.865043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":39474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9245","reported_seq":9663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.248822+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:39:34.248822+0000","last_peered":"2024-05-09T07:39:34.248822+0000","last_clean":"2024-05-09T07:39:34.248822+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:39:34.248822+0000","last_undegraded":"2024-05-09T07:39:34.248822+0000","last_fullsized":"2024-05-09T07:39:34.248822+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7223,"log_dups_size":0,"ondisk_log_size":7223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3175846,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":97790,"num_write":6533,"num_write_kb":843792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2528","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.812065+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:39:33.812065+0000","last_peered":"2024-05-09T07:39:33.812065+0000","last_clean":"2024-05-09T07:39:33.812065+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:39:33.812065+0000","last_undegraded":"2024-05-09T07:39:33.812065+0000","last_fullsized":"2024-05-09T07:39:33.812065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6992","reported_seq":7638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.782270+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:39:33.782270+0000","last_peered":"2024-05-09T07:39:33.782270+0000","last_clean":"2024-05-09T07:39:33.782270+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:39:33.782270+0000","last_undegraded":"2024-05-09T07:39:33.782270+0000","last_fullsized":"2024-05-09T07:39:33.782270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6992,"log_dups_size":0,"ondisk_log_size":6992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":608,"num_read_kb":4487,"num_write":9811,"num_write_kb":61141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9147","reported_seq":9330,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.892317+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:39:33.892317+0000","last_peered":"2024-05-09T07:39:33.892317+0000","last_clean":"2024-05-09T07:39:33.892317+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:39:33.892317+0000","last_undegraded":"2024-05-09T07:39:33.892317+0000","last_fullsized":"2024-05-09T07:39:33.892317+0000","mapping_epoch":22,"log_start":"27'2834","ondisk_log_start":"27'2834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1591544,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":150,"num_read_kb":38100,"num_write":6423,"num_write_kb":884596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2636","reported_seq":2674,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.131081+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:39:34.131081+0000","last_peered":"2024-05-09T07:39:34.131081+0000","last_clean":"2024-05-09T07:39:34.131081+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:39:34.131081+0000","last_undegraded":"2024-05-09T07:39:34.131081+0000","last_fullsized":"2024-05-09T07:39:34.131081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2636,"log_dups_size":0,"ondisk_log_size":2636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6519","reported_seq":6558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:29.207111+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:39:29.207111+0000","last_peered":"2024-05-09T07:39:29.207111+0000","last_clean":"2024-05-09T07:39:29.207111+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:39:29.207111+0000","last_undegraded":"2024-05-09T07:39:29.207111+0000","last_fullsized":"2024-05-09T07:39:29.207111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6519,"log_dups_size":0,"ondisk_log_size":6519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6522,"num_write_kb":53434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9269","reported_seq":10155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.551978+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:39:34.551978+0000","last_peered":"2024-05-09T07:39:34.551978+0000","last_clean":"2024-05-09T07:39:34.551978+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:39:34.551978+0000","last_undegraded":"2024-05-09T07:39:34.551978+0000","last_fullsized":"2024-05-09T07:39:34.551978+0000","mapping_epoch":22,"log_start":"27'1856","ondisk_log_start":"27'1856","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260096,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":853,"num_read_kb":216662,"num_write":6473,"num_write_kb":849337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2515","reported_seq":2537,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.642256+0000","last_change":"2024-05-09T07:14:26.645955+0000","last_active":"2024-05-09T07:39:33.642256+0000","last_peered":"2024-05-09T07:39:33.642256+0000","last_clean":"2024-05-09T07:39:33.642256+0000","last_became_active":"2024-05-09T07:14:26.645530+0000","last_became_peered":"2024-05-09T07:14:26.645530+0000","last_unstale":"2024-05-09T07:39:33.642256+0000","last_undegraded":"2024-05-09T07:39:33.642256+0000","last_fullsized":"2024-05-09T07:39:33.642256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'8371","reported_seq":8411,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:26.913891+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:39:26.913891+0000","last_peered":"2024-05-09T07:39:26.913891+0000","last_clean":"2024-05-09T07:39:26.913891+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:39:26.913891+0000","last_undegraded":"2024-05-09T07:39:26.913891+0000","last_fullsized":"2024-05-09T07:39:26.913891+0000","mapping_epoch":17,"log_start":"27'1000","ondisk_log_start":"27'1000","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8366,"num_write_kb":75026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9858","reported_seq":10329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.561712+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:39:34.561712+0000","last_peered":"2024-05-09T07:39:34.561712+0000","last_clean":"2024-05-09T07:39:34.561712+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:39:34.561712+0000","last_undegraded":"2024-05-09T07:39:34.561712+0000","last_fullsized":"2024-05-09T07:39:34.561712+0000","mapping_epoch":22,"log_start":"27'2476","ondisk_log_start":"27'2476","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":438,"num_read_kb":111252,"num_write":7195,"num_write_kb":963969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2563","reported_seq":2600,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.119968+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:39:34.119968+0000","last_peered":"2024-05-09T07:39:34.119968+0000","last_clean":"2024-05-09T07:39:34.119968+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:39:34.119968+0000","last_undegraded":"2024-05-09T07:39:34.119968+0000","last_fullsized":"2024-05-09T07:39:34.119968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2563,"log_dups_size":0,"ondisk_log_size":2563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'9194","reported_seq":9234,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:20.730299+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:39:20.730299+0000","last_peered":"2024-05-09T07:39:20.730299+0000","last_clean":"2024-05-09T07:39:20.730299+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:39:20.730299+0000","last_undegraded":"2024-05-09T07:39:20.730299+0000","last_fullsized":"2024-05-09T07:39:20.730299+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9195,"num_write_kb":63563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9946","reported_seq":10111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.331272+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:39:34.331272+0000","last_peered":"2024-05-09T07:39:34.331272+0000","last_clean":"2024-05-09T07:39:34.331272+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:39:34.331272+0000","last_undegraded":"2024-05-09T07:39:34.331272+0000","last_fullsized":"2024-05-09T07:39:34.331272+0000","mapping_epoch":22,"log_start":"27'2616","ondisk_log_start":"27'2616","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":33528,"num_write":7175,"num_write_kb":934104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2576","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.026717+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:39:34.026717+0000","last_peered":"2024-05-09T07:39:34.026717+0000","last_clean":"2024-05-09T07:39:34.026717+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:39:34.026717+0000","last_undegraded":"2024-05-09T07:39:34.026717+0000","last_fullsized":"2024-05-09T07:39:34.026717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7786","reported_seq":8725,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.660101+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:39:33.660101+0000","last_peered":"2024-05-09T07:39:33.660101+0000","last_clean":"2024-05-09T07:39:33.660101+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:39:33.660101+0000","last_undegraded":"2024-05-09T07:39:33.660101+0000","last_fullsized":"2024-05-09T07:39:33.660101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7786,"log_dups_size":0,"ondisk_log_size":7786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8410861,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":900,"num_read_kb":4675,"num_write":7771,"num_write_kb":62460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9564","reported_seq":9939,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.665427+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:39:34.665427+0000","last_peered":"2024-05-09T07:39:34.665427+0000","last_clean":"2024-05-09T07:39:34.665427+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:39:34.665427+0000","last_undegraded":"2024-05-09T07:39:34.665427+0000","last_fullsized":"2024-05-09T07:39:34.665427+0000","mapping_epoch":22,"log_start":"27'2181","ondisk_log_start":"27'2181","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":87122,"num_write":6858,"num_write_kb":899446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2521","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.239147+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:39:34.239147+0000","last_peered":"2024-05-09T07:39:34.239147+0000","last_clean":"2024-05-09T07:39:34.239147+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:39:34.239147+0000","last_undegraded":"2024-05-09T07:39:34.239147+0000","last_fullsized":"2024-05-09T07:39:34.239147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7846","reported_seq":7885,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:57.039762+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:38:57.039762+0000","last_peered":"2024-05-09T07:38:57.039762+0000","last_clean":"2024-05-09T07:38:57.039762+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:38:57.039762+0000","last_undegraded":"2024-05-09T07:38:57.039762+0000","last_fullsized":"2024-05-09T07:38:57.039762+0000","mapping_epoch":17,"log_start":"27'800","ondisk_log_start":"27'800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7848,"num_write_kb":58481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9410","reported_seq":9664,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.339794+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:39:34.339794+0000","last_peered":"2024-05-09T07:39:34.339794+0000","last_clean":"2024-05-09T07:39:34.339794+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:39:34.339794+0000","last_undegraded":"2024-05-09T07:39:34.339794+0000","last_fullsized":"2024-05-09T07:39:34.339794+0000","mapping_epoch":22,"log_start":"27'2042","ondisk_log_start":"27'2042","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":56388,"num_write":6716,"num_write_kb":1113422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2631","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.545182+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:39:33.545182+0000","last_peered":"2024-05-09T07:39:33.545182+0000","last_clean":"2024-05-09T07:39:33.545182+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:39:33.545182+0000","last_undegraded":"2024-05-09T07:39:33.545182+0000","last_fullsized":"2024-05-09T07:39:33.545182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2631,"log_dups_size":0,"ondisk_log_size":2631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4648","reported_seq":4688,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:53.493345+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:38:53.493345+0000","last_peered":"2024-05-09T07:38:53.493345+0000","last_clean":"2024-05-09T07:38:53.493345+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:38:53.493345+0000","last_undegraded":"2024-05-09T07:38:53.493345+0000","last_fullsized":"2024-05-09T07:38:53.493345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4648,"log_dups_size":0,"ondisk_log_size":4648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4652,"num_write_kb":39916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:35:39.524573+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:35:39.524573+0000","last_peered":"2024-05-09T07:35:39.524573+0000","last_clean":"2024-05-09T07:35:39.524573+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:35:39.524573+0000","last_undegraded":"2024-05-09T07:35:39.524573+0000","last_fullsized":"2024-05-09T07:35:39.524573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8966","reported_seq":9586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.979598+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:39:33.979598+0000","last_peered":"2024-05-09T07:39:33.979598+0000","last_clean":"2024-05-09T07:39:33.979598+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:39:33.979598+0000","last_undegraded":"2024-05-09T07:39:33.979598+0000","last_fullsized":"2024-05-09T07:39:33.979598+0000","mapping_epoch":22,"log_start":"27'1027","ondisk_log_start":"27'1027","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7939,"log_dups_size":0,"ondisk_log_size":7939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524831,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":587,"num_read_kb":149098,"num_write":6329,"num_write_kb":900398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2553","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.175294+0000","last_change":"2024-05-09T07:14:26.591809+0000","last_active":"2024-05-09T07:39:33.175294+0000","last_peered":"2024-05-09T07:39:33.175294+0000","last_clean":"2024-05-09T07:39:33.175294+0000","last_became_active":"2024-05-09T07:14:26.591447+0000","last_became_peered":"2024-05-09T07:14:26.591447+0000","last_unstale":"2024-05-09T07:39:33.175294+0000","last_undegraded":"2024-05-09T07:39:33.175294+0000","last_fullsized":"2024-05-09T07:39:33.175294+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"27'7870","reported_seq":7908,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:33.438472+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:38:33.438472+0000","last_peered":"2024-05-09T07:38:33.438472+0000","last_clean":"2024-05-09T07:38:33.438472+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:38:33.438472+0000","last_undegraded":"2024-05-09T07:38:33.438472+0000","last_fullsized":"2024-05-09T07:38:33.438472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7870,"log_dups_size":0,"ondisk_log_size":7870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7874,"num_write_kb":58494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9727","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.003249+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:39:34.003249+0000","last_peered":"2024-05-09T07:39:34.003249+0000","last_clean":"2024-05-09T07:39:34.003249+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:39:34.003249+0000","last_undegraded":"2024-05-09T07:39:34.003249+0000","last_fullsized":"2024-05-09T07:39:34.003249+0000","mapping_epoch":22,"log_start":"27'1734","ondisk_log_start":"27'1734","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7993,"log_dups_size":0,"ondisk_log_size":7993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976481,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":114046,"num_write":7013,"num_write_kb":962272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2581","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.572472+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:39:33.572472+0000","last_peered":"2024-05-09T07:39:33.572472+0000","last_clean":"2024-05-09T07:39:33.572472+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:39:33.572472+0000","last_undegraded":"2024-05-09T07:39:33.572472+0000","last_fullsized":"2024-05-09T07:39:33.572472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2581,"log_dups_size":0,"ondisk_log_size":2581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7541","reported_seq":7580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.569151+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:39:17.569151+0000","last_peered":"2024-05-09T07:39:17.569151+0000","last_clean":"2024-05-09T07:39:17.569151+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:39:17.569151+0000","last_undegraded":"2024-05-09T07:39:17.569151+0000","last_fullsized":"2024-05-09T07:39:17.569151+0000","mapping_epoch":17,"log_start":"27'200","ondisk_log_start":"27'200","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7542,"num_write_kb":58281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9828","reported_seq":10111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.629301+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:39:34.629301+0000","last_peered":"2024-05-09T07:39:34.629301+0000","last_clean":"2024-05-09T07:39:34.629301+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:39:34.629301+0000","last_undegraded":"2024-05-09T07:39:34.629301+0000","last_fullsized":"2024-05-09T07:39:34.629301+0000","mapping_epoch":22,"log_start":"27'2499","ondisk_log_start":"27'2499","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7329,"log_dups_size":0,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260096,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":251,"num_read_kb":63754,"num_write":7055,"num_write_kb":893799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2449","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.207708+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:39:34.207708+0000","last_peered":"2024-05-09T07:39:34.207708+0000","last_clean":"2024-05-09T07:39:34.207708+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:39:34.207708+0000","last_undegraded":"2024-05-09T07:39:34.207708+0000","last_fullsized":"2024-05-09T07:39:34.207708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8861","reported_seq":8899,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:36.701798+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:38:36.701798+0000","last_peered":"2024-05-09T07:38:36.701798+0000","last_clean":"2024-05-09T07:38:36.701798+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:38:36.701798+0000","last_undegraded":"2024-05-09T07:38:36.701798+0000","last_fullsized":"2024-05-09T07:38:36.701798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8861,"log_dups_size":0,"ondisk_log_size":8861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8852,"num_write_kb":79483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9353","reported_seq":10368,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.782422+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:39:33.782422+0000","last_peered":"2024-05-09T07:39:33.782422+0000","last_clean":"2024-05-09T07:39:33.782422+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:39:33.782422+0000","last_undegraded":"2024-05-09T07:39:33.782422+0000","last_fullsized":"2024-05-09T07:39:33.782422+0000","mapping_epoch":22,"log_start":"27'1982","ondisk_log_start":"27'1982","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4053700,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":983,"num_read_kb":249682,"num_write":6557,"num_write_kb":1037240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2553","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.169185+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:39:34.169185+0000","last_peered":"2024-05-09T07:39:34.169185+0000","last_clean":"2024-05-09T07:39:34.169185+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:39:34.169185+0000","last_undegraded":"2024-05-09T07:39:34.169185+0000","last_fullsized":"2024-05-09T07:39:34.169185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5464","reported_seq":5502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:48.455777+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:38:48.455777+0000","last_peered":"2024-05-09T07:38:48.455777+0000","last_clean":"2024-05-09T07:38:48.455777+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:38:48.455777+0000","last_undegraded":"2024-05-09T07:38:48.455777+0000","last_fullsized":"2024-05-09T07:38:48.455777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5457,"num_write_kb":44489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9303","reported_seq":9861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.254376+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:39:34.254376+0000","last_peered":"2024-05-09T07:39:34.254376+0000","last_clean":"2024-05-09T07:39:34.254376+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:39:34.254376+0000","last_undegraded":"2024-05-09T07:39:34.254376+0000","last_fullsized":"2024-05-09T07:39:34.254376+0000","mapping_epoch":22,"log_start":"27'2103","ondisk_log_start":"27'2103","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":527,"num_read_kb":133858,"num_write":6611,"num_write_kb":868619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2614","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.416883+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:39:34.416883+0000","last_peered":"2024-05-09T07:39:34.416883+0000","last_clean":"2024-05-09T07:39:34.416883+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:39:34.416883+0000","last_undegraded":"2024-05-09T07:39:34.416883+0000","last_fullsized":"2024-05-09T07:39:34.416883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2614,"log_dups_size":0,"ondisk_log_size":2614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3463","reported_seq":3501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:20.735079+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:39:20.735079+0000","last_peered":"2024-05-09T07:39:20.735079+0000","last_clean":"2024-05-09T07:39:20.735079+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:39:20.735079+0000","last_undegraded":"2024-05-09T07:39:20.735079+0000","last_fullsized":"2024-05-09T07:39:20.735079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3463,"log_dups_size":0,"ondisk_log_size":3463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3477,"num_write_kb":39113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9477","reported_seq":10052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.565062+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:39:34.565062+0000","last_peered":"2024-05-09T07:39:34.565062+0000","last_clean":"2024-05-09T07:39:34.565062+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:39:34.565062+0000","last_undegraded":"2024-05-09T07:39:34.565062+0000","last_fullsized":"2024-05-09T07:39:34.565062+0000","mapping_epoch":22,"log_start":"27'2114","ondisk_log_start":"27'2114","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":138176,"num_write":6755,"num_write_kb":1181456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2485","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.378977+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:39:33.378977+0000","last_peered":"2024-05-09T07:39:33.378977+0000","last_clean":"2024-05-09T07:39:33.378977+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:39:33.378977+0000","last_undegraded":"2024-05-09T07:39:33.378977+0000","last_fullsized":"2024-05-09T07:39:33.378977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4963","reported_seq":5002,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.760052+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:39:17.760052+0000","last_peered":"2024-05-09T07:39:17.760052+0000","last_clean":"2024-05-09T07:39:17.760052+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:39:17.760052+0000","last_undegraded":"2024-05-09T07:39:17.760052+0000","last_fullsized":"2024-05-09T07:39:17.760052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4963,"log_dups_size":0,"ondisk_log_size":4963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":4974,"num_write_kb":44143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9304","reported_seq":9607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.999932+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:39:33.999932+0000","last_peered":"2024-05-09T07:39:33.999932+0000","last_clean":"2024-05-09T07:39:33.999932+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:39:33.999932+0000","last_undegraded":"2024-05-09T07:39:33.999932+0000","last_fullsized":"2024-05-09T07:39:33.999932+0000","mapping_epoch":22,"log_start":"27'3038","ondisk_log_start":"27'3038","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6266,"log_dups_size":0,"ondisk_log_size":6266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4623858,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":272,"num_read_kb":69088,"num_write":6503,"num_write_kb":924175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2595","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.358643+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:39:34.358643+0000","last_peered":"2024-05-09T07:39:34.358643+0000","last_clean":"2024-05-09T07:39:34.358643+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:39:34.358643+0000","last_undegraded":"2024-05-09T07:39:34.358643+0000","last_fullsized":"2024-05-09T07:39:34.358643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2649","reported_seq":2688,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:14.327549+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:39:14.327549+0000","last_peered":"2024-05-09T07:39:14.327549+0000","last_clean":"2024-05-09T07:39:14.327549+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:39:14.327549+0000","last_undegraded":"2024-05-09T07:39:14.327549+0000","last_fullsized":"2024-05-09T07:39:14.327549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2650,"num_write_kb":30266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9217","reported_seq":9444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.929998+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:39:33.929998+0000","last_peered":"2024-05-09T07:39:33.929998+0000","last_clean":"2024-05-09T07:39:33.929998+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:39:33.929998+0000","last_undegraded":"2024-05-09T07:39:33.929998+0000","last_fullsized":"2024-05-09T07:39:33.929998+0000","mapping_epoch":22,"log_start":"27'2939","ondisk_log_start":"27'2939","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6278,"log_dups_size":0,"ondisk_log_size":6278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519806,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":197,"num_read_kb":50038,"num_write":6657,"num_write_kb":910804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2552","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.373769+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:39:34.373769+0000","last_peered":"2024-05-09T07:39:34.373769+0000","last_clean":"2024-05-09T07:39:34.373769+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:39:34.373769+0000","last_undegraded":"2024-05-09T07:39:34.373769+0000","last_fullsized":"2024-05-09T07:39:34.373769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2552,"log_dups_size":0,"ondisk_log_size":2552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6995","reported_seq":7033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:29.888509+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:39:29.888509+0000","last_peered":"2024-05-09T07:39:29.888509+0000","last_clean":"2024-05-09T07:39:29.888509+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:39:29.888509+0000","last_undegraded":"2024-05-09T07:39:29.888509+0000","last_fullsized":"2024-05-09T07:39:29.888509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6995,"log_dups_size":0,"ondisk_log_size":6995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6987,"num_write_kb":57820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9727","reported_seq":9995,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.789422+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:39:33.789422+0000","last_peered":"2024-05-09T07:39:33.789422+0000","last_clean":"2024-05-09T07:39:33.789422+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:39:33.789422+0000","last_undegraded":"2024-05-09T07:39:33.789422+0000","last_fullsized":"2024-05-09T07:39:33.789422+0000","mapping_epoch":22,"log_start":"27'2359","ondisk_log_start":"27'2359","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2496915,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":237,"num_read_kb":60198,"num_write":7040,"num_write_kb":1050276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2497","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.331398+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:39:33.331398+0000","last_peered":"2024-05-09T07:39:33.331398+0000","last_clean":"2024-05-09T07:39:33.331398+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:39:33.331398+0000","last_undegraded":"2024-05-09T07:39:33.331398+0000","last_fullsized":"2024-05-09T07:39:33.331398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10540","reported_seq":11205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.746477+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:39:17.746477+0000","last_peered":"2024-05-09T07:39:17.746477+0000","last_clean":"2024-05-09T07:39:17.746477+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:39:17.746477+0000","last_undegraded":"2024-05-09T07:39:17.746477+0000","last_fullsized":"2024-05-09T07:39:17.746477+0000","mapping_epoch":17,"log_start":"27'2600","ondisk_log_start":"27'2600","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7940,"log_dups_size":0,"ondisk_log_size":7940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":627,"num_read_kb":4520,"num_write":10533,"num_write_kb":76795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9186","reported_seq":9429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.849229+0000","last_change":"2024-05-09T07:14:26.592176+0000","last_active":"2024-05-09T07:39:33.849229+0000","last_peered":"2024-05-09T07:39:33.849229+0000","last_clean":"2024-05-09T07:39:33.849229+0000","last_became_active":"2024-05-09T07:14:26.590721+0000","last_became_peered":"2024-05-09T07:14:26.590721+0000","last_unstale":"2024-05-09T07:39:33.849229+0000","last_undegraded":"2024-05-09T07:39:33.849229+0000","last_fullsized":"2024-05-09T07:39:33.849229+0000","mapping_epoch":26,"log_start":"27'1828","ondisk_log_start":"27'1828","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:38.451332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4278543,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":221,"num_read_kb":56134,"num_write":6500,"num_write_kb":978701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"27'2569","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.706846+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:39:33.706846+0000","last_peered":"2024-05-09T07:39:33.706846+0000","last_clean":"2024-05-09T07:39:33.706846+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:39:33.706846+0000","last_undegraded":"2024-05-09T07:39:33.706846+0000","last_fullsized":"2024-05-09T07:39:33.706846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6174","reported_seq":6212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:27.333611+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:39:27.333611+0000","last_peered":"2024-05-09T07:39:27.333611+0000","last_clean":"2024-05-09T07:39:27.333611+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:39:27.333611+0000","last_undegraded":"2024-05-09T07:39:27.333611+0000","last_fullsized":"2024-05-09T07:39:27.333611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6180,"num_write_kb":41394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9167","reported_seq":9510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.231279+0000","last_change":"2024-05-09T07:14:26.596397+0000","last_active":"2024-05-09T07:39:34.231279+0000","last_peered":"2024-05-09T07:39:34.231279+0000","last_clean":"2024-05-09T07:39:34.231279+0000","last_became_active":"2024-05-09T07:14:26.595936+0000","last_became_peered":"2024-05-09T07:14:26.595936+0000","last_unstale":"2024-05-09T07:39:34.231279+0000","last_undegraded":"2024-05-09T07:39:34.231279+0000","last_fullsized":"2024-05-09T07:39:34.231279+0000","mapping_epoch":26,"log_start":"27'2010","ondisk_log_start":"27'2010","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7157,"log_dups_size":0,"ondisk_log_size":7157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:45.938473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040384,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":305,"num_read_kb":77470,"num_write":6442,"num_write_kb":989255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2602","reported_seq":2637,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.576974+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:39:34.576974+0000","last_peered":"2024-05-09T07:39:34.576974+0000","last_clean":"2024-05-09T07:39:34.576974+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:39:34.576974+0000","last_undegraded":"2024-05-09T07:39:34.576974+0000","last_fullsized":"2024-05-09T07:39:34.576974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2602,"log_dups_size":0,"ondisk_log_size":2602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5738","reported_seq":5776,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:06.691749+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:39:06.691749+0000","last_peered":"2024-05-09T07:39:06.691749+0000","last_clean":"2024-05-09T07:39:06.691749+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:39:06.691749+0000","last_undegraded":"2024-05-09T07:39:06.691749+0000","last_fullsized":"2024-05-09T07:39:06.691749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5739,"num_write_kb":52939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9112","reported_seq":9479,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.730627+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:39:33.730627+0000","last_peered":"2024-05-09T07:39:33.730627+0000","last_clean":"2024-05-09T07:39:33.730627+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:39:33.730627+0000","last_undegraded":"2024-05-09T07:39:33.730627+0000","last_fullsized":"2024-05-09T07:39:33.730627+0000","mapping_epoch":22,"log_start":"27'2805","ondisk_log_start":"27'2805","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6307,"log_dups_size":0,"ondisk_log_size":6307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1304576,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":335,"num_read_kb":85090,"num_write":6440,"num_write_kb":967374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2569","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.769929+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:39:33.769929+0000","last_peered":"2024-05-09T07:39:33.769929+0000","last_clean":"2024-05-09T07:39:33.769929+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:39:33.769929+0000","last_undegraded":"2024-05-09T07:39:33.769929+0000","last_fullsized":"2024-05-09T07:39:33.769929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'5942","reported_seq":6846,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:15.634681+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:38:15.634681+0000","last_peered":"2024-05-09T07:38:15.634681+0000","last_clean":"2024-05-09T07:38:15.634681+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:38:15.634681+0000","last_undegraded":"2024-05-09T07:38:15.634681+0000","last_fullsized":"2024-05-09T07:38:15.634681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":867,"num_read_kb":4659,"num_write":5943,"num_write_kb":49048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9593","reported_seq":9981,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.605018+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:39:33.605018+0000","last_peered":"2024-05-09T07:39:33.605018+0000","last_clean":"2024-05-09T07:39:33.605018+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:39:33.605018+0000","last_undegraded":"2024-05-09T07:39:33.605018+0000","last_fullsized":"2024-05-09T07:39:33.605018+0000","mapping_epoch":22,"log_start":"27'3323","ondisk_log_start":"27'3323","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746850,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6812,"num_write_kb":1059940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9162","reported_seq":9389,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.603712+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:39:34.603712+0000","last_peered":"2024-05-09T07:39:34.603712+0000","last_clean":"2024-05-09T07:39:34.603712+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:39:34.603712+0000","last_undegraded":"2024-05-09T07:39:34.603712+0000","last_fullsized":"2024-05-09T07:39:34.603712+0000","mapping_epoch":22,"log_start":"27'1813","ondisk_log_start":"27'1813","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7349,"log_dups_size":0,"ondisk_log_size":7349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":49276,"num_write":6559,"num_write_kb":912347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2528","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.594958+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:39:34.594958+0000","last_peered":"2024-05-09T07:39:34.594958+0000","last_clean":"2024-05-09T07:39:34.594958+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:39:34.594958+0000","last_undegraded":"2024-05-09T07:39:34.594958+0000","last_fullsized":"2024-05-09T07:39:34.594958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9067","reported_seq":9105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:08.929933+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:38:08.929933+0000","last_peered":"2024-05-09T07:38:08.929933+0000","last_clean":"2024-05-09T07:38:08.929933+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:38:08.929933+0000","last_undegraded":"2024-05-09T07:38:08.929933+0000","last_fullsized":"2024-05-09T07:38:08.929933+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9064,"num_write_kb":67566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8837","reported_seq":10023,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.965223+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:39:33.965223+0000","last_peered":"2024-05-09T07:39:33.965223+0000","last_clean":"2024-05-09T07:39:33.965223+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:39:33.965223+0000","last_undegraded":"2024-05-09T07:39:33.965223+0000","last_fullsized":"2024-05-09T07:39:33.965223+0000","mapping_epoch":22,"log_start":"27'900","ondisk_log_start":"27'900","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7937,"log_dups_size":0,"ondisk_log_size":7937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1948104,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1153,"num_read_kb":292862,"num_write":6075,"num_write_kb":877058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2618","reported_seq":2654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.701610+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:39:33.701610+0000","last_peered":"2024-05-09T07:39:33.701610+0000","last_clean":"2024-05-09T07:39:33.701610+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:39:33.701610+0000","last_undegraded":"2024-05-09T07:39:33.701610+0000","last_fullsized":"2024-05-09T07:39:33.701610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2618,"log_dups_size":0,"ondisk_log_size":2618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5353","reported_seq":5391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:13.460729+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:38:13.460729+0000","last_peered":"2024-05-09T07:38:13.460729+0000","last_clean":"2024-05-09T07:38:13.460729+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:38:13.460729+0000","last_undegraded":"2024-05-09T07:38:13.460729+0000","last_fullsized":"2024-05-09T07:38:13.460729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5353,"log_dups_size":0,"ondisk_log_size":5353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5353,"num_write_kb":44481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9693","reported_seq":9731,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:43.717707+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:38:43.717707+0000","last_peered":"2024-05-09T07:38:43.717707+0000","last_clean":"2024-05-09T07:38:43.717707+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:38:43.717707+0000","last_undegraded":"2024-05-09T07:38:43.717707+0000","last_fullsized":"2024-05-09T07:38:43.717707+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7993,"log_dups_size":0,"ondisk_log_size":7993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9690,"num_write_kb":72019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2555","reported_seq":2590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.417888+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:39:34.417888+0000","last_peered":"2024-05-09T07:39:34.417888+0000","last_clean":"2024-05-09T07:39:34.417888+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:39:34.417888+0000","last_undegraded":"2024-05-09T07:39:34.417888+0000","last_fullsized":"2024-05-09T07:39:34.417888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9392","reported_seq":9447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.677629+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:39:33.677629+0000","last_peered":"2024-05-09T07:39:33.677629+0000","last_clean":"2024-05-09T07:39:33.677629+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:39:33.677629+0000","last_undegraded":"2024-05-09T07:39:33.677629+0000","last_fullsized":"2024-05-09T07:39:33.677629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1565758,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":24,"num_read_kb":6096,"num_write":6594,"num_write_kb":983207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8024","reported_seq":8062,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:48.275994+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:38:48.275994+0000","last_peered":"2024-05-09T07:38:48.275994+0000","last_clean":"2024-05-09T07:38:48.275994+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:38:48.275994+0000","last_undegraded":"2024-05-09T07:38:48.275994+0000","last_fullsized":"2024-05-09T07:38:48.275994+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6324,"log_dups_size":0,"ondisk_log_size":6324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8026,"num_write_kb":66635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2597","reported_seq":2632,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.242028+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:39:34.242028+0000","last_peered":"2024-05-09T07:39:34.242028+0000","last_clean":"2024-05-09T07:39:34.242028+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:39:34.242028+0000","last_undegraded":"2024-05-09T07:39:34.242028+0000","last_fullsized":"2024-05-09T07:39:34.242028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9823","reported_seq":9975,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.979566+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:39:33.979566+0000","last_peered":"2024-05-09T07:39:33.979566+0000","last_clean":"2024-05-09T07:39:33.979566+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:39:33.979566+0000","last_undegraded":"2024-05-09T07:39:33.979566+0000","last_fullsized":"2024-05-09T07:39:33.979566+0000","mapping_epoch":22,"log_start":"27'3543","ondisk_log_start":"27'3543","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081890,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":121,"num_read_kb":30734,"num_write":7010,"num_write_kb":1030226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2491","reported_seq":2529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.847333+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:39:33.847333+0000","last_peered":"2024-05-09T07:39:33.847333+0000","last_clean":"2024-05-09T07:39:33.847333+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:39:33.847333+0000","last_undegraded":"2024-05-09T07:39:33.847333+0000","last_fullsized":"2024-05-09T07:39:33.847333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2491,"log_dups_size":0,"ondisk_log_size":2491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2611","reported_seq":2648,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.123638+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:39:33.123638+0000","last_peered":"2024-05-09T07:39:33.123638+0000","last_clean":"2024-05-09T07:39:33.123638+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:39:33.123638+0000","last_undegraded":"2024-05-09T07:39:33.123638+0000","last_fullsized":"2024-05-09T07:39:33.123638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2515","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.622088+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:39:33.622088+0000","last_peered":"2024-05-09T07:39:33.622088+0000","last_clean":"2024-05-09T07:39:33.622088+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:39:33.622088+0000","last_undegraded":"2024-05-09T07:39:33.622088+0000","last_fullsized":"2024-05-09T07:39:33.622088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2604","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.371669+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:39:34.371669+0000","last_peered":"2024-05-09T07:39:34.371669+0000","last_clean":"2024-05-09T07:39:34.371669+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:39:34.371669+0000","last_undegraded":"2024-05-09T07:39:34.371669+0000","last_fullsized":"2024-05-09T07:39:34.371669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2604,"log_dups_size":0,"ondisk_log_size":2604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2596","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.755716+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:39:33.755716+0000","last_peered":"2024-05-09T07:39:33.755716+0000","last_clean":"2024-05-09T07:39:33.755716+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:39:33.755716+0000","last_undegraded":"2024-05-09T07:39:33.755716+0000","last_fullsized":"2024-05-09T07:39:33.755716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2482","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.054999+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:39:34.054999+0000","last_peered":"2024-05-09T07:39:34.054999+0000","last_clean":"2024-05-09T07:39:34.054999+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:39:34.054999+0000","last_undegraded":"2024-05-09T07:39:34.054999+0000","last_fullsized":"2024-05-09T07:39:34.054999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2610","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.002968+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:39:34.002968+0000","last_peered":"2024-05-09T07:39:34.002968+0000","last_clean":"2024-05-09T07:39:34.002968+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:39:34.002968+0000","last_undegraded":"2024-05-09T07:39:34.002968+0000","last_fullsized":"2024-05-09T07:39:34.002968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2610,"log_dups_size":0,"ondisk_log_size":2610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2582","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.942776+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:39:33.942776+0000","last_peered":"2024-05-09T07:39:33.942776+0000","last_clean":"2024-05-09T07:39:33.942776+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:39:33.942776+0000","last_undegraded":"2024-05-09T07:39:33.942776+0000","last_fullsized":"2024-05-09T07:39:33.942776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2566","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.702866+0000","last_change":"2024-05-09T07:14:26.646404+0000","last_active":"2024-05-09T07:39:33.702866+0000","last_peered":"2024-05-09T07:39:33.702866+0000","last_clean":"2024-05-09T07:39:33.702866+0000","last_became_active":"2024-05-09T07:14:26.646058+0000","last_became_peered":"2024-05-09T07:14:26.646058+0000","last_unstale":"2024-05-09T07:39:33.702866+0000","last_undegraded":"2024-05-09T07:39:33.702866+0000","last_fullsized":"2024-05-09T07:39:33.702866+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2566,"log_dups_size":0,"ondisk_log_size":2566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'2525","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.518809+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:39:33.518809+0000","last_peered":"2024-05-09T07:39:33.518809+0000","last_clean":"2024-05-09T07:39:33.518809+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:39:33.518809+0000","last_undegraded":"2024-05-09T07:39:33.518809+0000","last_fullsized":"2024-05-09T07:39:33.518809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2599","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.840412+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:39:33.840412+0000","last_peered":"2024-05-09T07:39:33.840412+0000","last_clean":"2024-05-09T07:39:33.840412+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:39:33.840412+0000","last_undegraded":"2024-05-09T07:39:33.840412+0000","last_fullsized":"2024-05-09T07:39:33.840412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2608","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.225626+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:39:34.225626+0000","last_peered":"2024-05-09T07:39:34.225626+0000","last_clean":"2024-05-09T07:39:34.225626+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:39:34.225626+0000","last_undegraded":"2024-05-09T07:39:34.225626+0000","last_fullsized":"2024-05-09T07:39:34.225626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2601","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:32.409252+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:39:32.409252+0000","last_peered":"2024-05-09T07:39:32.409252+0000","last_clean":"2024-05-09T07:39:32.409252+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:39:32.409252+0000","last_undegraded":"2024-05-09T07:39:32.409252+0000","last_fullsized":"2024-05-09T07:39:32.409252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2606","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.135665+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:39:34.135665+0000","last_peered":"2024-05-09T07:39:34.135665+0000","last_clean":"2024-05-09T07:39:34.135665+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:39:34.135665+0000","last_undegraded":"2024-05-09T07:39:34.135665+0000","last_fullsized":"2024-05-09T07:39:34.135665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2544","reported_seq":2580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.079417+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:39:33.079417+0000","last_peered":"2024-05-09T07:39:33.079417+0000","last_clean":"2024-05-09T07:39:33.079417+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:39:33.079417+0000","last_undegraded":"2024-05-09T07:39:33.079417+0000","last_fullsized":"2024-05-09T07:39:33.079417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2609","reported_seq":2646,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.792677+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:39:33.792677+0000","last_peered":"2024-05-09T07:39:33.792677+0000","last_clean":"2024-05-09T07:39:33.792677+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:39:33.792677+0000","last_undegraded":"2024-05-09T07:39:33.792677+0000","last_fullsized":"2024-05-09T07:39:33.792677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2609,"log_dups_size":0,"ondisk_log_size":2609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2547","reported_seq":2584,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.736964+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:39:33.736964+0000","last_peered":"2024-05-09T07:39:33.736964+0000","last_clean":"2024-05-09T07:39:33.736964+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:39:33.736964+0000","last_undegraded":"2024-05-09T07:39:33.736964+0000","last_fullsized":"2024-05-09T07:39:33.736964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2547,"log_dups_size":0,"ondisk_log_size":2547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2669","reported_seq":2706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.483917+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:39:34.483917+0000","last_peered":"2024-05-09T07:39:34.483917+0000","last_clean":"2024-05-09T07:39:34.483917+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:39:34.483917+0000","last_undegraded":"2024-05-09T07:39:34.483917+0000","last_fullsized":"2024-05-09T07:39:34.483917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2669,"log_dups_size":0,"ondisk_log_size":2669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2520","reported_seq":2556,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.617853+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:39:34.617853+0000","last_peered":"2024-05-09T07:39:34.617853+0000","last_clean":"2024-05-09T07:39:34.617853+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:39:34.617853+0000","last_undegraded":"2024-05-09T07:39:34.617853+0000","last_fullsized":"2024-05-09T07:39:34.617853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2504","reported_seq":2540,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.803579+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:39:33.803579+0000","last_peered":"2024-05-09T07:39:33.803579+0000","last_clean":"2024-05-09T07:39:33.803579+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:39:33.803579+0000","last_undegraded":"2024-05-09T07:39:33.803579+0000","last_fullsized":"2024-05-09T07:39:33.803579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2504,"log_dups_size":0,"ondisk_log_size":2504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2482","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.172679+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:39:34.172679+0000","last_peered":"2024-05-09T07:39:34.172679+0000","last_clean":"2024-05-09T07:39:34.172679+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:39:34.172679+0000","last_undegraded":"2024-05-09T07:39:34.172679+0000","last_fullsized":"2024-05-09T07:39:34.172679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2503","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.838109+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:39:33.838109+0000","last_peered":"2024-05-09T07:39:33.838109+0000","last_clean":"2024-05-09T07:39:33.838109+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:39:33.838109+0000","last_undegraded":"2024-05-09T07:39:33.838109+0000","last_fullsized":"2024-05-09T07:39:33.838109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2632","reported_seq":2668,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.753412+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:39:33.753412+0000","last_peered":"2024-05-09T07:39:33.753412+0000","last_clean":"2024-05-09T07:39:33.753412+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:39:33.753412+0000","last_undegraded":"2024-05-09T07:39:33.753412+0000","last_fullsized":"2024-05-09T07:39:33.753412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2632,"log_dups_size":0,"ondisk_log_size":2632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2599","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.441391+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:39:34.441391+0000","last_peered":"2024-05-09T07:39:34.441391+0000","last_clean":"2024-05-09T07:39:34.441391+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:39:34.441391+0000","last_undegraded":"2024-05-09T07:39:34.441391+0000","last_fullsized":"2024-05-09T07:39:34.441391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2537","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.832419+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:39:33.832419+0000","last_peered":"2024-05-09T07:39:33.832419+0000","last_clean":"2024-05-09T07:39:33.832419+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:39:33.832419+0000","last_undegraded":"2024-05-09T07:39:33.832419+0000","last_fullsized":"2024-05-09T07:39:33.832419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2537,"log_dups_size":0,"ondisk_log_size":2537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.487057+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:39:34.487057+0000","last_peered":"2024-05-09T07:39:34.487057+0000","last_clean":"2024-05-09T07:39:34.487057+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:39:34.487057+0000","last_undegraded":"2024-05-09T07:39:34.487057+0000","last_fullsized":"2024-05-09T07:39:34.487057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2503","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.726849+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:39:33.726849+0000","last_peered":"2024-05-09T07:39:33.726849+0000","last_clean":"2024-05-09T07:39:33.726849+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:39:33.726849+0000","last_undegraded":"2024-05-09T07:39:33.726849+0000","last_fullsized":"2024-05-09T07:39:33.726849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2542","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.644003+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:39:34.644003+0000","last_peered":"2024-05-09T07:39:34.644003+0000","last_clean":"2024-05-09T07:39:34.644003+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:39:34.644003+0000","last_undegraded":"2024-05-09T07:39:34.644003+0000","last_fullsized":"2024-05-09T07:39:34.644003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2580","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.459093+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:39:33.459093+0000","last_peered":"2024-05-09T07:39:33.459093+0000","last_clean":"2024-05-09T07:39:33.459093+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:39:33.459093+0000","last_undegraded":"2024-05-09T07:39:33.459093+0000","last_fullsized":"2024-05-09T07:39:33.459093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2658","reported_seq":2695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.225175+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:39:34.225175+0000","last_peered":"2024-05-09T07:39:34.225175+0000","last_clean":"2024-05-09T07:39:34.225175+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:39:34.225175+0000","last_undegraded":"2024-05-09T07:39:34.225175+0000","last_fullsized":"2024-05-09T07:39:34.225175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2496","reported_seq":2534,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.141290+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:39:34.141290+0000","last_peered":"2024-05-09T07:39:34.141290+0000","last_clean":"2024-05-09T07:39:34.141290+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:39:34.141290+0000","last_undegraded":"2024-05-09T07:39:34.141290+0000","last_fullsized":"2024-05-09T07:39:34.141290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2482","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.536252+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:39:34.536252+0000","last_peered":"2024-05-09T07:39:34.536252+0000","last_clean":"2024-05-09T07:39:34.536252+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:39:34.536252+0000","last_undegraded":"2024-05-09T07:39:34.536252+0000","last_fullsized":"2024-05-09T07:39:34.536252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":61502438,"num_objects":168,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":12270,"num_read_kb":3117350,"num_write":214562,"num_write_kb":30722105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120573952,"data_stored":120573952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232059,"ondisk_log_size":232059,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164175,"ondisk_log_size":164175,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":248505113,"num_objects":195,"num_object_clones":0,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":195,"num_whiteouts":0,"num_read":3177,"num_read_kb":19424,"num_write":211653,"num_write_kb":1695367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":127,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497180672,"data_stored":497116653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192901,"ondisk_log_size":192901,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575163,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":970376,"kb_used_data":64640,"kb_used_omap":1,"kb_used_meta":905726,"kb_avail":93401464,"statfs":{"total":96636764160,"available":95643099136,"internally_reserved":0,"allocated":66191360,"data_stored":66012455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":927463882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575162,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1129536,"kb_used_data":91960,"kb_used_omap":1,"kb_used_meta":1037566,"kb_avail":93242304,"statfs":{"total":96636764160,"available":95480119296,"internally_reserved":0,"allocated":94167040,"data_stored":93987681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1062468038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575162,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1019696,"kb_used_data":81896,"kb_used_omap":1,"kb_used_meta":937662,"kb_avail":93352144,"statfs":{"total":96636764160,"available":95592595456,"internally_reserved":0,"allocated":83861504,"data_stored":83687882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960166341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575161,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062008,"kb_used_data":84016,"kb_used_omap":1,"kb_used_meta":977854,"kb_avail":93309832,"statfs":{"total":96636764160,"available":95549267968,"internally_reserved":0,"allocated":86032384,"data_stored":85850833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001322950},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575163,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279116,"kb_used_data":128116,"kb_used_omap":1,"kb_used_meta":1150078,"kb_avail":93092724,"statfs":{"total":96636764160,"available":95326949376,"internally_reserved":0,"allocated":131190784,"data_stored":131013875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1177680327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729932,"kb_used_data":76932,"kb_used_omap":1,"kb_used_meta":652990,"kb_avail":93641908,"statfs":{"total":96636764160,"available":95889313792,"internally_reserved":0,"allocated":78778368,"data_stored":78606656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668662213},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575165,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1251056,"kb_used_data":82072,"kb_used_omap":1,"kb_used_meta":1168062,"kb_avail":93120784,"statfs":{"total":96636764160,"available":95355682816,"internally_reserved":0,"allocated":84041728,"data_stored":83864307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1196095941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898152,"kb_used_data":68896,"kb_used_omap":1,"kb_used_meta":829246,"kb_avail":93473688,"statfs":{"total":96636764160,"available":95717056512,"internally_reserved":0,"allocated":70549504,"data_stored":70364913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":849148357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67109502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54583296,"data_stored":54582378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71409664,"data_stored":71398347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46141440,"data_stored":46137434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":68136960,"data_stored":68126990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":55549952,"data_stored":55543988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2760704,"data_stored":2760704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23805952,"data_stored":23805952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29478912,"data_stored":29478912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16297984,"data_stored":16297984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21856256,"data_stored":21856256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24752128,"data_stored":24752128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":573440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:39:38.784 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-09T07:39:38.784 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:39:38.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.912+0000 7f44c5763640 1 Processor -- start 2024-05-09T07:39:38.923 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.912+0000 7f44c5763640 1 -- start start 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44c5763640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f44c006e400 0x7f44c00c1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44c5763640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 0x7f44c006e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44c5763640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f44c0155bf0 con 0x7f44c0155350 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44c5763640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f44c0155ed0 con 0x7f44c006e400 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44c5763640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44c01561b0 con 0x7f44c006b1f0 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44b7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 0x7f44c006e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.925 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44b7fff640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 0x7f44c006e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:35344/0 (socket says 172.21.15.160:35344) 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44b7fff640 1 -- 172.21.15.160:0/341623213 learned_addr learned my addr 172.21.15.160:0/341623213 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.913+0000 7f44beffd640 1 --2- 172.21.15.160:0/341623213 >> v2:172.21.15.188:3300/0 conn(0x7f44c006e400 0x7f44c00c1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.914+0000 7f44b7fff640 1 -- 172.21.15.160:0/341623213 >> v2:172.21.15.188:3300/0 conn(0x7f44c006e400 msgr2=0x7f44c00c1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.914+0000 7f44b7fff640 1 --2- 172.21.15.160:0/341623213 >> v2:172.21.15.188:3300/0 conn(0x7f44c006e400 0x7f44c00c1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.914+0000 7f44b7fff640 1 -- 172.21.15.160:0/341623213 >> v1:172.21.15.188:6789/0 conn(0x7f44c0155350 legacy=0x7f44c0159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.914+0000 7f44b7fff640 1 -- 172.21.15.160:0/341623213 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c0158940 con 0x7f44c006b1f0 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.914+0000 7f44beffd640 1 --2- 172.21.15.160:0/341623213 >> v2:172.21.15.188:3300/0 conn(0x7f44c006e400 0x7f44c00c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.914+0000 7f44b7fff640 1 --2- 172.21.15.160:0/341623213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 0x7f44c006e7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009870 tx=0x7f44a802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f61d47f82060361 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44be7fc640 1 -- 172.21.15.160:0/341623213 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8037070 con 0x7f44c006b1f0 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44be7fc640 1 -- 172.21.15.160:0/341623213 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44a8032440 con 0x7f44c006b1f0 2024-05-09T07:39:38.926 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44be7fc640 1 -- 172.21.15.160:0/341623213 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8032810 con 0x7f44c006b1f0 2024-05-09T07:39:38.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 -- 172.21.15.160:0/341623213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 msgr2=0x7f44c006e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.928 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 --2- 172.21.15.160:0/341623213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 0x7f44c006e7e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009870 tx=0x7f44a802a1c0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 -- 172.21.15.160:0/341623213 shutdown_connections 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 --2- 172.21.15.160:0/341623213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006b1f0 0x7f44c006e7e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 --2- 172.21.15.160:0/341623213 >> v2:172.21.15.188:3300/0 conn(0x7f44c006e400 0x7f44c00c1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 -- 172.21.15.160:0/341623213 >> 172.21.15.160:0/341623213 conn(0x7f44c00c76e0 msgr2=0x7f44c00c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 -- 172.21.15.160:0/341623213 shutdown_connections 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.915+0000 7f44c5763640 1 -- 172.21.15.160:0/341623213 wait complete. 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.916+0000 7f44c5763640 1 Processor -- start 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.916+0000 7f44c5763640 1 -- start start 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44c5763640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c006b1f0 0x7f44c01ee3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44c5763640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006e400 0x7f44c014a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44c5763640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 0x7f44c014a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44c5763640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01557e0 con 0x7f44c0155350 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44c5763640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0159850 con 0x7f44c006b1f0 2024-05-09T07:39:38.929 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44c5763640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f44c0155960 con 0x7f44c006e400 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 0x7f44c014a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 0x7f44c014a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34712/0 (socket says 172.21.15.160:34712) 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 -- 172.21.15.160:0/3874942691 learned_addr learned my addr 172.21.15.160:0/3874942691 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44b7fff640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006e400 0x7f44c014a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44beffd640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c006b1f0 0x7f44c01ee3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 -- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006e400 msgr2=0x7f44c014a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006e400 0x7f44c014a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 -- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c006b1f0 msgr2=0x7f44c01ee3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c006b1f0 0x7f44c01ee3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.917+0000 7f44bf7fe640 1 -- 172.21.15.160:0/3874942691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c00c6d90 con 0x7f44c0155350 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.918+0000 7f44bf7fe640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 0x7f44c014a870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f44b800c7a0 tx=0x7f44b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.918+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8003b70 con 0x7f44c0155350 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.918+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44b8003d30 con 0x7f44c0155350 2024-05-09T07:39:38.930 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.918+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b800aa90 con 0x7f44c0155350 2024-05-09T07:39:38.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.918+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c01eec30 con 0x7f44c0155350 2024-05-09T07:39:38.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.918+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c00c7070 con 0x7f44c0155350 2024-05-09T07:39:38.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.919+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44c00756c0 con 0x7f44c0155350 2024-05-09T07:39:38.931 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.920+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f44b800ac30 con 0x7f44c0155350 2024-05-09T07:39:38.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.920+0000 7f44b77fe640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f44a006be30 0x7f44a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:38.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.920+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f44b808afd0 con 0x7f44c0155350 2024-05-09T07:39:38.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.920+0000 7f44beffd640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f44a006be30 0x7f44a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:38.932 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.921+0000 7f44beffd640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f44a006be30 0x7f44a006e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009560 tx=0x7f44b0009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:38.935 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:38.924+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f44b8058fd0 con 0x7f44c0155350 2024-05-09T07:39:39.118 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.107+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44c00675a0 con 0x7f44a006be30 2024-05-09T07:39:39.126 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.115+0000 7f44b77fe640 1 -- 172.21.15.160:0/3874942691 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336227 (secure 0 0 0) 0x7f44c00675a0 con 0x7f44a006be30 2024-05-09T07:39:39.129 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:39:39.132 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:39:39.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f44a006be30 msgr2=0x7f44a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.133 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f44a006be30 0x7f44a006e2f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009560 tx=0x7f44b0009190 comp rx=0 tx=0).stop 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 msgr2=0x7f44c014a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 0x7f44c014a870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f44b800c7a0 tx=0x7f44b800cc60 comp rx=0 tx=0).stop 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 shutdown_connections 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f44a006be30 0x7f44a006e2f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f44c0155350 0x7f44c014a870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f44c006e400 0x7f44c014a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 --2- 172.21.15.160:0/3874942691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f44c006b1f0 0x7f44c01ee3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 >> 172.21.15.160:0/3874942691 conn(0x7f44c00c76e0 msgr2=0x7f44c014e000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 shutdown_connections 2024-05-09T07:39:39.134 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.119+0000 7f44c5763640 1 -- 172.21.15.160:0/3874942691 wait complete. 2024-05-09T07:39:39.141 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":802,"stamp":"2024-05-09T07:39:37.561520+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":310597919,"num_objects":478,"num_object_clones":0,"num_object_copies":1292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":478,"num_whiteouts":0,"num_read":15563,"num_read_kb":3136874,"num_write":427381,"num_write_kb":32419888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":127,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":589252,"ondisk_log_size":589252,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8339872,"kb_used_data":678528,"kb_used_omap":12,"kb_used_meta":7659187,"kb_avail":746634848,"statfs":{"total":773094113280,"available":764554084352,"internally_reserved":0,"allocated":694812672,"data_stored":693388602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":7843008049},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,6,4,12,18,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-148056247,"num_objects":-1063,"num_object_clones":0,"num_object_copies":-3364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1063,"num_whiteouts":0,"num_read":84,"num_read_kb":17248,"num_write":3915,"num_write_kb":86342,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003252"},"pg_stats":[{"pgid":"2.1f","version":"27'2672","reported_seq":2711,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:51.735149+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:38:51.735149+0000","last_peered":"2024-05-09T07:38:51.735149+0000","last_clean":"2024-05-09T07:38:51.735149+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:38:51.735149+0000","last_undegraded":"2024-05-09T07:38:51.735149+0000","last_fullsized":"2024-05-09T07:38:51.735149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":2684,"num_write_kb":18241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2569","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.929399+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:39:33.929399+0000","last_peered":"2024-05-09T07:39:33.929399+0000","last_clean":"2024-05-09T07:39:33.929399+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:39:33.929399+0000","last_undegraded":"2024-05-09T07:39:33.929399+0000","last_fullsized":"2024-05-09T07:39:33.929399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9807","reported_seq":10237,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.981916+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:39:33.981916+0000","last_peered":"2024-05-09T07:39:33.981916+0000","last_clean":"2024-05-09T07:39:33.981916+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:39:33.981916+0000","last_undegraded":"2024-05-09T07:39:33.981916+0000","last_fullsized":"2024-05-09T07:39:33.981916+0000","mapping_epoch":22,"log_start":"27'3504","ondisk_log_start":"27'3504","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6303,"log_dups_size":0,"ondisk_log_size":6303,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6872883,"num_objects":16,"num_object_clones":0,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":399,"num_read_kb":101346,"num_write":7160,"num_write_kb":1009567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'7784","reported_seq":7824,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:23.622641+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:39:23.622641+0000","last_peered":"2024-05-09T07:39:23.622641+0000","last_clean":"2024-05-09T07:39:23.622641+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:39:23.622641+0000","last_undegraded":"2024-05-09T07:39:23.622641+0000","last_fullsized":"2024-05-09T07:39:23.622641+0000","mapping_epoch":17,"log_start":"27'1500","ondisk_log_start":"27'1500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":7796,"num_write_kb":54453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2550","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.085746+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:39:34.085746+0000","last_peered":"2024-05-09T07:39:34.085746+0000","last_clean":"2024-05-09T07:39:34.085746+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:39:34.085746+0000","last_undegraded":"2024-05-09T07:39:34.085746+0000","last_fullsized":"2024-05-09T07:39:34.085746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9531","reported_seq":9918,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.127850+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:39:34.127850+0000","last_peered":"2024-05-09T07:39:34.127850+0000","last_clean":"2024-05-09T07:39:34.127850+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:39:34.127850+0000","last_undegraded":"2024-05-09T07:39:34.127850+0000","last_fullsized":"2024-05-09T07:39:34.127850+0000","mapping_epoch":22,"log_start":"27'2329","ondisk_log_start":"27'2329","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6759,"num_write_kb":1095702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9522","reported_seq":9695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.191922+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:39:33.191922+0000","last_peered":"2024-05-09T07:39:33.191922+0000","last_clean":"2024-05-09T07:39:33.191922+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:39:33.191922+0000","last_undegraded":"2024-05-09T07:39:33.191922+0000","last_fullsized":"2024-05-09T07:39:33.191922+0000","mapping_epoch":17,"log_start":"27'2500","ondisk_log_start":"27'2500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7022,"log_dups_size":0,"ondisk_log_size":7022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5211758,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":138,"num_read_kb":1083,"num_write":9520,"num_write_kb":77042,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2657","reported_seq":2693,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.656593+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:39:33.656593+0000","last_peered":"2024-05-09T07:39:33.656593+0000","last_clean":"2024-05-09T07:39:33.656593+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:39:33.656593+0000","last_undegraded":"2024-05-09T07:39:33.656593+0000","last_fullsized":"2024-05-09T07:39:33.656593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2657,"log_dups_size":0,"ondisk_log_size":2657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9063","reported_seq":9472,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.136390+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:39:34.136390+0000","last_peered":"2024-05-09T07:39:34.136390+0000","last_clean":"2024-05-09T07:39:34.136390+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:39:34.136390+0000","last_undegraded":"2024-05-09T07:39:34.136390+0000","last_fullsized":"2024-05-09T07:39:34.136390+0000","mapping_epoch":22,"log_start":"27'1901","ondisk_log_start":"27'1901","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5917224,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6298,"num_write_kb":900735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4288","reported_seq":4325,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.588906+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:39:33.588906+0000","last_peered":"2024-05-09T07:39:33.588906+0000","last_clean":"2024-05-09T07:39:33.588906+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:39:33.588906+0000","last_undegraded":"2024-05-09T07:39:33.588906+0000","last_fullsized":"2024-05-09T07:39:33.588906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4288,"log_dups_size":0,"ondisk_log_size":4288,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":4301,"num_write_kb":47774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2587","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.753943+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:39:33.753943+0000","last_peered":"2024-05-09T07:39:33.753943+0000","last_clean":"2024-05-09T07:39:33.753943+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:39:33.753943+0000","last_undegraded":"2024-05-09T07:39:33.753943+0000","last_fullsized":"2024-05-09T07:39:33.753943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'9248","reported_seq":9721,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.271401+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:39:34.271401+0000","last_peered":"2024-05-09T07:39:34.271401+0000","last_clean":"2024-05-09T07:39:34.271401+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:39:34.271401+0000","last_undegraded":"2024-05-09T07:39:34.271401+0000","last_fullsized":"2024-05-09T07:39:34.271401+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7226,"log_dups_size":0,"ondisk_log_size":7226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1308672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":442,"num_read_kb":112268,"num_write":6537,"num_write_kb":853848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2578","reported_seq":2615,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.930484+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:39:33.930484+0000","last_peered":"2024-05-09T07:39:33.930484+0000","last_clean":"2024-05-09T07:39:33.930484+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:39:33.930484+0000","last_undegraded":"2024-05-09T07:39:33.930484+0000","last_fullsized":"2024-05-09T07:39:33.930484+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6961","reported_seq":6999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:03.399820+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:39:03.399820+0000","last_peered":"2024-05-09T07:39:03.399820+0000","last_clean":"2024-05-09T07:39:03.399820+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:39:03.399820+0000","last_undegraded":"2024-05-09T07:39:03.399820+0000","last_fullsized":"2024-05-09T07:39:03.399820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6961,"log_dups_size":0,"ondisk_log_size":6961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6969,"num_write_kb":53856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9806","reported_seq":10053,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.172117+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:39:34.172117+0000","last_peered":"2024-05-09T07:39:34.172117+0000","last_clean":"2024-05-09T07:39:34.172117+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:39:34.172117+0000","last_undegraded":"2024-05-09T07:39:34.172117+0000","last_fullsized":"2024-05-09T07:39:34.172117+0000","mapping_epoch":22,"log_start":"27'2633","ondisk_log_start":"27'2633","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":215,"num_read_kb":54610,"num_write":7039,"num_write_kb":1008697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2599","reported_seq":2636,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.034015+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:39:34.034015+0000","last_peered":"2024-05-09T07:39:34.034015+0000","last_clean":"2024-05-09T07:39:34.034015+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:39:34.034015+0000","last_undegraded":"2024-05-09T07:39:34.034015+0000","last_fullsized":"2024-05-09T07:39:34.034015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7778","reported_seq":7817,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:42.793042+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:38:42.793042+0000","last_peered":"2024-05-09T07:38:42.793042+0000","last_clean":"2024-05-09T07:38:42.793042+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:38:42.793042+0000","last_undegraded":"2024-05-09T07:38:42.793042+0000","last_fullsized":"2024-05-09T07:38:42.793042+0000","mapping_epoch":17,"log_start":"27'400","ondisk_log_start":"27'400","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7378,"log_dups_size":0,"ondisk_log_size":7378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7776,"num_write_kb":58435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9627","reported_seq":10167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.260189+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:39:34.260189+0000","last_peered":"2024-05-09T07:39:34.260189+0000","last_clean":"2024-05-09T07:39:34.260189+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:39:34.260189+0000","last_undegraded":"2024-05-09T07:39:34.260189+0000","last_fullsized":"2024-05-09T07:39:34.260189+0000","mapping_epoch":22,"log_start":"27'2443","ondisk_log_start":"27'2443","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7184,"log_dups_size":0,"ondisk_log_size":7184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2628189,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":508,"num_read_kb":129802,"num_write":6847,"num_write_kb":970185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2672","reported_seq":2709,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.851191+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:39:33.851191+0000","last_peered":"2024-05-09T07:39:33.851191+0000","last_clean":"2024-05-09T07:39:33.851191+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:39:33.851191+0000","last_undegraded":"2024-05-09T07:39:33.851191+0000","last_fullsized":"2024-05-09T07:39:33.851191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2672,"log_dups_size":0,"ondisk_log_size":2672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3774","reported_seq":3813,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:40.266938+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:37:40.266938+0000","last_peered":"2024-05-09T07:37:40.266938+0000","last_clean":"2024-05-09T07:37:40.266938+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:37:40.266938+0000","last_undegraded":"2024-05-09T07:37:40.266938+0000","last_fullsized":"2024-05-09T07:37:40.266938+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3774,"log_dups_size":0,"ondisk_log_size":3774,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3771,"num_write_kb":27117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9040","reported_seq":9652,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.430298+0000","last_change":"2024-05-09T07:14:26.646797+0000","last_active":"2024-05-09T07:39:33.430298+0000","last_peered":"2024-05-09T07:39:33.430298+0000","last_clean":"2024-05-09T07:39:33.430298+0000","last_became_active":"2024-05-09T07:14:26.646222+0000","last_became_peered":"2024-05-09T07:14:26.646222+0000","last_unstale":"2024-05-09T07:39:33.430298+0000","last_undegraded":"2024-05-09T07:39:33.430298+0000","last_fullsized":"2024-05-09T07:39:33.430298+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9040,"log_dups_size":0,"ondisk_log_size":9040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3553201,"num_objects":13,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6364,"num_write_kb":893211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2580","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.213138+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:39:34.213138+0000","last_peered":"2024-05-09T07:39:34.213138+0000","last_clean":"2024-05-09T07:39:34.213138+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:39:34.213138+0000","last_undegraded":"2024-05-09T07:39:34.213138+0000","last_fullsized":"2024-05-09T07:39:34.213138+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2263","reported_seq":2301,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:10.954485+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:37:10.954485+0000","last_peered":"2024-05-09T07:37:10.954485+0000","last_clean":"2024-05-09T07:37:10.954485+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:37:10.954485+0000","last_undegraded":"2024-05-09T07:37:10.954485+0000","last_fullsized":"2024-05-09T07:37:10.954485+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2272,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9928","reported_seq":10064,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.216405+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:39:34.216405+0000","last_peered":"2024-05-09T07:39:34.216405+0000","last_clean":"2024-05-09T07:39:34.216405+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:39:34.216405+0000","last_undegraded":"2024-05-09T07:39:34.216405+0000","last_fullsized":"2024-05-09T07:39:34.216405+0000","mapping_epoch":22,"log_start":"27'2728","ondisk_log_start":"27'2728","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1371263,"num_objects":3,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":26416,"num_write":7233,"num_write_kb":974347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2532","reported_seq":2568,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.954387+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:39:33.954387+0000","last_peered":"2024-05-09T07:39:33.954387+0000","last_clean":"2024-05-09T07:39:33.954387+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:39:33.954387+0000","last_undegraded":"2024-05-09T07:39:33.954387+0000","last_fullsized":"2024-05-09T07:39:33.954387+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2532,"log_dups_size":0,"ondisk_log_size":2532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4066","reported_seq":4105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:39.865043+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:38:39.865043+0000","last_peered":"2024-05-09T07:38:39.865043+0000","last_clean":"2024-05-09T07:38:39.865043+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:38:39.865043+0000","last_undegraded":"2024-05-09T07:38:39.865043+0000","last_fullsized":"2024-05-09T07:38:39.865043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":39474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9245","reported_seq":9663,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.248822+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:39:34.248822+0000","last_peered":"2024-05-09T07:39:34.248822+0000","last_clean":"2024-05-09T07:39:34.248822+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:39:34.248822+0000","last_undegraded":"2024-05-09T07:39:34.248822+0000","last_fullsized":"2024-05-09T07:39:34.248822+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7223,"log_dups_size":0,"ondisk_log_size":7223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3175846,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":385,"num_read_kb":97790,"num_write":6533,"num_write_kb":843792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2528","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.812065+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:39:33.812065+0000","last_peered":"2024-05-09T07:39:33.812065+0000","last_clean":"2024-05-09T07:39:33.812065+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:39:33.812065+0000","last_undegraded":"2024-05-09T07:39:33.812065+0000","last_fullsized":"2024-05-09T07:39:33.812065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6992","reported_seq":7638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.782270+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:39:33.782270+0000","last_peered":"2024-05-09T07:39:33.782270+0000","last_clean":"2024-05-09T07:39:33.782270+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:39:33.782270+0000","last_undegraded":"2024-05-09T07:39:33.782270+0000","last_fullsized":"2024-05-09T07:39:33.782270+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6992,"log_dups_size":0,"ondisk_log_size":6992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":608,"num_read_kb":4487,"num_write":9811,"num_write_kb":61141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9147","reported_seq":9330,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.892317+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:39:33.892317+0000","last_peered":"2024-05-09T07:39:33.892317+0000","last_clean":"2024-05-09T07:39:33.892317+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:39:33.892317+0000","last_undegraded":"2024-05-09T07:39:33.892317+0000","last_fullsized":"2024-05-09T07:39:33.892317+0000","mapping_epoch":22,"log_start":"27'2834","ondisk_log_start":"27'2834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6313,"log_dups_size":0,"ondisk_log_size":6313,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1591544,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":150,"num_read_kb":38100,"num_write":6423,"num_write_kb":884596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2636","reported_seq":2674,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.131081+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:39:34.131081+0000","last_peered":"2024-05-09T07:39:34.131081+0000","last_clean":"2024-05-09T07:39:34.131081+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:39:34.131081+0000","last_undegraded":"2024-05-09T07:39:34.131081+0000","last_fullsized":"2024-05-09T07:39:34.131081+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2636,"log_dups_size":0,"ondisk_log_size":2636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6519","reported_seq":6558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:29.207111+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:39:29.207111+0000","last_peered":"2024-05-09T07:39:29.207111+0000","last_clean":"2024-05-09T07:39:29.207111+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:39:29.207111+0000","last_undegraded":"2024-05-09T07:39:29.207111+0000","last_fullsized":"2024-05-09T07:39:29.207111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6519,"log_dups_size":0,"ondisk_log_size":6519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6522,"num_write_kb":53434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9269","reported_seq":10155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.551978+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:39:34.551978+0000","last_peered":"2024-05-09T07:39:34.551978+0000","last_clean":"2024-05-09T07:39:34.551978+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:39:34.551978+0000","last_undegraded":"2024-05-09T07:39:34.551978+0000","last_fullsized":"2024-05-09T07:39:34.551978+0000","mapping_epoch":22,"log_start":"27'1856","ondisk_log_start":"27'1856","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7413,"log_dups_size":0,"ondisk_log_size":7413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260096,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":853,"num_read_kb":216662,"num_write":6473,"num_write_kb":849337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2515","reported_seq":2537,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.642256+0000","last_change":"2024-05-09T07:14:26.645955+0000","last_active":"2024-05-09T07:39:33.642256+0000","last_peered":"2024-05-09T07:39:33.642256+0000","last_clean":"2024-05-09T07:39:33.642256+0000","last_became_active":"2024-05-09T07:14:26.645530+0000","last_became_peered":"2024-05-09T07:14:26.645530+0000","last_unstale":"2024-05-09T07:39:33.642256+0000","last_undegraded":"2024-05-09T07:39:33.642256+0000","last_fullsized":"2024-05-09T07:39:33.642256+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'8371","reported_seq":8411,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:26.913891+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:39:26.913891+0000","last_peered":"2024-05-09T07:39:26.913891+0000","last_clean":"2024-05-09T07:39:26.913891+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:39:26.913891+0000","last_undegraded":"2024-05-09T07:39:26.913891+0000","last_fullsized":"2024-05-09T07:39:26.913891+0000","mapping_epoch":17,"log_start":"27'1000","ondisk_log_start":"27'1000","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8366,"num_write_kb":75026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9858","reported_seq":10329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.561712+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:39:34.561712+0000","last_peered":"2024-05-09T07:39:34.561712+0000","last_clean":"2024-05-09T07:39:34.561712+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:39:34.561712+0000","last_undegraded":"2024-05-09T07:39:34.561712+0000","last_fullsized":"2024-05-09T07:39:34.561712+0000","mapping_epoch":22,"log_start":"27'2476","ondisk_log_start":"27'2476","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":438,"num_read_kb":111252,"num_write":7195,"num_write_kb":963969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2563","reported_seq":2600,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.119968+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:39:34.119968+0000","last_peered":"2024-05-09T07:39:34.119968+0000","last_clean":"2024-05-09T07:39:34.119968+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:39:34.119968+0000","last_undegraded":"2024-05-09T07:39:34.119968+0000","last_fullsized":"2024-05-09T07:39:34.119968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2563,"log_dups_size":0,"ondisk_log_size":2563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'9194","reported_seq":9234,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:20.730299+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:39:20.730299+0000","last_peered":"2024-05-09T07:39:20.730299+0000","last_clean":"2024-05-09T07:39:20.730299+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:39:20.730299+0000","last_undegraded":"2024-05-09T07:39:20.730299+0000","last_fullsized":"2024-05-09T07:39:20.730299+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7394,"log_dups_size":0,"ondisk_log_size":7394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9195,"num_write_kb":63563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9946","reported_seq":10111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.331272+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:39:34.331272+0000","last_peered":"2024-05-09T07:39:34.331272+0000","last_clean":"2024-05-09T07:39:34.331272+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:39:34.331272+0000","last_undegraded":"2024-05-09T07:39:34.331272+0000","last_fullsized":"2024-05-09T07:39:34.331272+0000","mapping_epoch":22,"log_start":"27'2616","ondisk_log_start":"27'2616","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":33528,"num_write":7175,"num_write_kb":934104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2576","reported_seq":2614,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.026717+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:39:34.026717+0000","last_peered":"2024-05-09T07:39:34.026717+0000","last_clean":"2024-05-09T07:39:34.026717+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:39:34.026717+0000","last_undegraded":"2024-05-09T07:39:34.026717+0000","last_fullsized":"2024-05-09T07:39:34.026717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7786","reported_seq":8725,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.660101+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:39:33.660101+0000","last_peered":"2024-05-09T07:39:33.660101+0000","last_clean":"2024-05-09T07:39:33.660101+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:39:33.660101+0000","last_undegraded":"2024-05-09T07:39:33.660101+0000","last_fullsized":"2024-05-09T07:39:33.660101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7786,"log_dups_size":0,"ondisk_log_size":7786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8410861,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":900,"num_read_kb":4675,"num_write":7771,"num_write_kb":62460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9564","reported_seq":9939,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.665427+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:39:34.665427+0000","last_peered":"2024-05-09T07:39:34.665427+0000","last_clean":"2024-05-09T07:39:34.665427+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:39:34.665427+0000","last_undegraded":"2024-05-09T07:39:34.665427+0000","last_fullsized":"2024-05-09T07:39:34.665427+0000","mapping_epoch":22,"log_start":"27'2181","ondisk_log_start":"27'2181","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":87122,"num_write":6858,"num_write_kb":899446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2521","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.239147+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:39:34.239147+0000","last_peered":"2024-05-09T07:39:34.239147+0000","last_clean":"2024-05-09T07:39:34.239147+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:39:34.239147+0000","last_undegraded":"2024-05-09T07:39:34.239147+0000","last_fullsized":"2024-05-09T07:39:34.239147+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7846","reported_seq":7885,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:57.039762+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:38:57.039762+0000","last_peered":"2024-05-09T07:38:57.039762+0000","last_clean":"2024-05-09T07:38:57.039762+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:38:57.039762+0000","last_undegraded":"2024-05-09T07:38:57.039762+0000","last_fullsized":"2024-05-09T07:38:57.039762+0000","mapping_epoch":17,"log_start":"27'800","ondisk_log_start":"27'800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7046,"log_dups_size":0,"ondisk_log_size":7046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7848,"num_write_kb":58481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9410","reported_seq":9664,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.339794+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:39:34.339794+0000","last_peered":"2024-05-09T07:39:34.339794+0000","last_clean":"2024-05-09T07:39:34.339794+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:39:34.339794+0000","last_undegraded":"2024-05-09T07:39:34.339794+0000","last_fullsized":"2024-05-09T07:39:34.339794+0000","mapping_epoch":22,"log_start":"27'2042","ondisk_log_start":"27'2042","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":520192,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":222,"num_read_kb":56388,"num_write":6716,"num_write_kb":1113422,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2631","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.545182+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:39:33.545182+0000","last_peered":"2024-05-09T07:39:33.545182+0000","last_clean":"2024-05-09T07:39:33.545182+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:39:33.545182+0000","last_undegraded":"2024-05-09T07:39:33.545182+0000","last_fullsized":"2024-05-09T07:39:33.545182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2631,"log_dups_size":0,"ondisk_log_size":2631,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4648","reported_seq":4688,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:53.493345+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:38:53.493345+0000","last_peered":"2024-05-09T07:38:53.493345+0000","last_clean":"2024-05-09T07:38:53.493345+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:38:53.493345+0000","last_undegraded":"2024-05-09T07:38:53.493345+0000","last_fullsized":"2024-05-09T07:38:53.493345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4648,"log_dups_size":0,"ondisk_log_size":4648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4652,"num_write_kb":39916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":195,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:35:39.524573+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:35:39.524573+0000","last_peered":"2024-05-09T07:35:39.524573+0000","last_clean":"2024-05-09T07:35:39.524573+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:35:39.524573+0000","last_undegraded":"2024-05-09T07:35:39.524573+0000","last_fullsized":"2024-05-09T07:35:39.524573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8966","reported_seq":9586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.979598+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:39:33.979598+0000","last_peered":"2024-05-09T07:39:33.979598+0000","last_clean":"2024-05-09T07:39:33.979598+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:39:33.979598+0000","last_undegraded":"2024-05-09T07:39:33.979598+0000","last_fullsized":"2024-05-09T07:39:33.979598+0000","mapping_epoch":22,"log_start":"27'1027","ondisk_log_start":"27'1027","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7939,"log_dups_size":0,"ondisk_log_size":7939,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":524831,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":587,"num_read_kb":149098,"num_write":6329,"num_write_kb":900398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2553","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.175294+0000","last_change":"2024-05-09T07:14:26.591809+0000","last_active":"2024-05-09T07:39:33.175294+0000","last_peered":"2024-05-09T07:39:33.175294+0000","last_clean":"2024-05-09T07:39:33.175294+0000","last_became_active":"2024-05-09T07:14:26.591447+0000","last_became_peered":"2024-05-09T07:14:26.591447+0000","last_unstale":"2024-05-09T07:39:33.175294+0000","last_undegraded":"2024-05-09T07:39:33.175294+0000","last_fullsized":"2024-05-09T07:39:33.175294+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"27'7870","reported_seq":7908,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:33.438472+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:38:33.438472+0000","last_peered":"2024-05-09T07:38:33.438472+0000","last_clean":"2024-05-09T07:38:33.438472+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:38:33.438472+0000","last_undegraded":"2024-05-09T07:38:33.438472+0000","last_fullsized":"2024-05-09T07:38:33.438472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7870,"log_dups_size":0,"ondisk_log_size":7870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7874,"num_write_kb":58494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9727","reported_seq":10208,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.003249+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:39:34.003249+0000","last_peered":"2024-05-09T07:39:34.003249+0000","last_clean":"2024-05-09T07:39:34.003249+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:39:34.003249+0000","last_undegraded":"2024-05-09T07:39:34.003249+0000","last_fullsized":"2024-05-09T07:39:34.003249+0000","mapping_epoch":22,"log_start":"27'1734","ondisk_log_start":"27'1734","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7993,"log_dups_size":0,"ondisk_log_size":7993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1976481,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":449,"num_read_kb":114046,"num_write":7013,"num_write_kb":962272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2581","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.572472+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:39:33.572472+0000","last_peered":"2024-05-09T07:39:33.572472+0000","last_clean":"2024-05-09T07:39:33.572472+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:39:33.572472+0000","last_undegraded":"2024-05-09T07:39:33.572472+0000","last_fullsized":"2024-05-09T07:39:33.572472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2581,"log_dups_size":0,"ondisk_log_size":2581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7541","reported_seq":7580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.569151+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:39:17.569151+0000","last_peered":"2024-05-09T07:39:17.569151+0000","last_clean":"2024-05-09T07:39:17.569151+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:39:17.569151+0000","last_undegraded":"2024-05-09T07:39:17.569151+0000","last_fullsized":"2024-05-09T07:39:17.569151+0000","mapping_epoch":17,"log_start":"27'200","ondisk_log_start":"27'200","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7341,"log_dups_size":0,"ondisk_log_size":7341,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7542,"num_write_kb":58281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9828","reported_seq":10111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.629301+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:39:34.629301+0000","last_peered":"2024-05-09T07:39:34.629301+0000","last_clean":"2024-05-09T07:39:34.629301+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:39:34.629301+0000","last_undegraded":"2024-05-09T07:39:34.629301+0000","last_fullsized":"2024-05-09T07:39:34.629301+0000","mapping_epoch":22,"log_start":"27'2499","ondisk_log_start":"27'2499","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7329,"log_dups_size":0,"ondisk_log_size":7329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260096,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":251,"num_read_kb":63754,"num_write":7055,"num_write_kb":893799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2449","reported_seq":2485,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.207708+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:39:34.207708+0000","last_peered":"2024-05-09T07:39:34.207708+0000","last_clean":"2024-05-09T07:39:34.207708+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:39:34.207708+0000","last_undegraded":"2024-05-09T07:39:34.207708+0000","last_fullsized":"2024-05-09T07:39:34.207708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2449,"log_dups_size":0,"ondisk_log_size":2449,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8861","reported_seq":8899,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:36.701798+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:38:36.701798+0000","last_peered":"2024-05-09T07:38:36.701798+0000","last_clean":"2024-05-09T07:38:36.701798+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:38:36.701798+0000","last_undegraded":"2024-05-09T07:38:36.701798+0000","last_fullsized":"2024-05-09T07:38:36.701798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8861,"log_dups_size":0,"ondisk_log_size":8861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8852,"num_write_kb":79483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9353","reported_seq":10368,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.782422+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:39:33.782422+0000","last_peered":"2024-05-09T07:39:33.782422+0000","last_clean":"2024-05-09T07:39:33.782422+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:39:33.782422+0000","last_undegraded":"2024-05-09T07:39:33.782422+0000","last_fullsized":"2024-05-09T07:39:33.782422+0000","mapping_epoch":22,"log_start":"27'1982","ondisk_log_start":"27'1982","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4053700,"num_objects":11,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":983,"num_read_kb":249682,"num_write":6557,"num_write_kb":1037240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2553","reported_seq":2589,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.169185+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:39:34.169185+0000","last_peered":"2024-05-09T07:39:34.169185+0000","last_clean":"2024-05-09T07:39:34.169185+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:39:34.169185+0000","last_undegraded":"2024-05-09T07:39:34.169185+0000","last_fullsized":"2024-05-09T07:39:34.169185+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5464","reported_seq":5502,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:48.455777+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:38:48.455777+0000","last_peered":"2024-05-09T07:38:48.455777+0000","last_clean":"2024-05-09T07:38:48.455777+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:38:48.455777+0000","last_undegraded":"2024-05-09T07:38:48.455777+0000","last_fullsized":"2024-05-09T07:38:48.455777+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5464,"log_dups_size":0,"ondisk_log_size":5464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5457,"num_write_kb":44489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9303","reported_seq":9861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.254376+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:39:34.254376+0000","last_peered":"2024-05-09T07:39:34.254376+0000","last_clean":"2024-05-09T07:39:34.254376+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:39:34.254376+0000","last_undegraded":"2024-05-09T07:39:34.254376+0000","last_fullsized":"2024-05-09T07:39:34.254376+0000","mapping_epoch":22,"log_start":"27'2103","ondisk_log_start":"27'2103","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7200,"log_dups_size":0,"ondisk_log_size":7200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1052672,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":527,"num_read_kb":133858,"num_write":6611,"num_write_kb":868619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2614","reported_seq":2650,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.416883+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:39:34.416883+0000","last_peered":"2024-05-09T07:39:34.416883+0000","last_clean":"2024-05-09T07:39:34.416883+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:39:34.416883+0000","last_undegraded":"2024-05-09T07:39:34.416883+0000","last_fullsized":"2024-05-09T07:39:34.416883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2614,"log_dups_size":0,"ondisk_log_size":2614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3463","reported_seq":3501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:20.735079+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:39:20.735079+0000","last_peered":"2024-05-09T07:39:20.735079+0000","last_clean":"2024-05-09T07:39:20.735079+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:39:20.735079+0000","last_undegraded":"2024-05-09T07:39:20.735079+0000","last_fullsized":"2024-05-09T07:39:20.735079+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3463,"log_dups_size":0,"ondisk_log_size":3463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3477,"num_write_kb":39113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9477","reported_seq":10052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.565062+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:39:34.565062+0000","last_peered":"2024-05-09T07:39:34.565062+0000","last_clean":"2024-05-09T07:39:34.565062+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:39:34.565062+0000","last_undegraded":"2024-05-09T07:39:34.565062+0000","last_fullsized":"2024-05-09T07:39:34.565062+0000","mapping_epoch":22,"log_start":"27'2114","ondisk_log_start":"27'2114","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":138176,"num_write":6755,"num_write_kb":1181456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2485","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.378977+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:39:33.378977+0000","last_peered":"2024-05-09T07:39:33.378977+0000","last_clean":"2024-05-09T07:39:33.378977+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:39:33.378977+0000","last_undegraded":"2024-05-09T07:39:33.378977+0000","last_fullsized":"2024-05-09T07:39:33.378977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4963","reported_seq":5002,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.760052+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:39:17.760052+0000","last_peered":"2024-05-09T07:39:17.760052+0000","last_clean":"2024-05-09T07:39:17.760052+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:39:17.760052+0000","last_undegraded":"2024-05-09T07:39:17.760052+0000","last_fullsized":"2024-05-09T07:39:17.760052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4963,"log_dups_size":0,"ondisk_log_size":4963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":4974,"num_write_kb":44143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9304","reported_seq":9607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.999932+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:39:33.999932+0000","last_peered":"2024-05-09T07:39:33.999932+0000","last_clean":"2024-05-09T07:39:33.999932+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:39:33.999932+0000","last_undegraded":"2024-05-09T07:39:33.999932+0000","last_fullsized":"2024-05-09T07:39:33.999932+0000","mapping_epoch":22,"log_start":"27'3038","ondisk_log_start":"27'3038","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6266,"log_dups_size":0,"ondisk_log_size":6266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4623858,"num_objects":14,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":272,"num_read_kb":69088,"num_write":6503,"num_write_kb":924175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2595","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.358643+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:39:34.358643+0000","last_peered":"2024-05-09T07:39:34.358643+0000","last_clean":"2024-05-09T07:39:34.358643+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:39:34.358643+0000","last_undegraded":"2024-05-09T07:39:34.358643+0000","last_fullsized":"2024-05-09T07:39:34.358643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2595,"log_dups_size":0,"ondisk_log_size":2595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2649","reported_seq":2688,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:14.327549+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:39:14.327549+0000","last_peered":"2024-05-09T07:39:14.327549+0000","last_clean":"2024-05-09T07:39:14.327549+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:39:14.327549+0000","last_undegraded":"2024-05-09T07:39:14.327549+0000","last_fullsized":"2024-05-09T07:39:14.327549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2649,"log_dups_size":0,"ondisk_log_size":2649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2650,"num_write_kb":30266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9217","reported_seq":9444,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.929998+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:39:33.929998+0000","last_peered":"2024-05-09T07:39:33.929998+0000","last_clean":"2024-05-09T07:39:33.929998+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:39:33.929998+0000","last_undegraded":"2024-05-09T07:39:33.929998+0000","last_fullsized":"2024-05-09T07:39:33.929998+0000","mapping_epoch":22,"log_start":"27'2939","ondisk_log_start":"27'2939","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6278,"log_dups_size":0,"ondisk_log_size":6278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519806,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":197,"num_read_kb":50038,"num_write":6657,"num_write_kb":910804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2552","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.373769+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:39:34.373769+0000","last_peered":"2024-05-09T07:39:34.373769+0000","last_clean":"2024-05-09T07:39:34.373769+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:39:34.373769+0000","last_undegraded":"2024-05-09T07:39:34.373769+0000","last_fullsized":"2024-05-09T07:39:34.373769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2552,"log_dups_size":0,"ondisk_log_size":2552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6995","reported_seq":7033,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:29.888509+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:39:29.888509+0000","last_peered":"2024-05-09T07:39:29.888509+0000","last_clean":"2024-05-09T07:39:29.888509+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:39:29.888509+0000","last_undegraded":"2024-05-09T07:39:29.888509+0000","last_fullsized":"2024-05-09T07:39:29.888509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6995,"log_dups_size":0,"ondisk_log_size":6995,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6987,"num_write_kb":57820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9727","reported_seq":9995,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.789422+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:39:33.789422+0000","last_peered":"2024-05-09T07:39:33.789422+0000","last_clean":"2024-05-09T07:39:33.789422+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:39:33.789422+0000","last_undegraded":"2024-05-09T07:39:33.789422+0000","last_fullsized":"2024-05-09T07:39:33.789422+0000","mapping_epoch":22,"log_start":"27'2359","ondisk_log_start":"27'2359","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2496915,"num_objects":5,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":237,"num_read_kb":60198,"num_write":7040,"num_write_kb":1050276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2497","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.331398+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:39:33.331398+0000","last_peered":"2024-05-09T07:39:33.331398+0000","last_clean":"2024-05-09T07:39:33.331398+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:39:33.331398+0000","last_undegraded":"2024-05-09T07:39:33.331398+0000","last_fullsized":"2024-05-09T07:39:33.331398+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2497,"log_dups_size":0,"ondisk_log_size":2497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10540","reported_seq":11205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.746477+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:39:17.746477+0000","last_peered":"2024-05-09T07:39:17.746477+0000","last_clean":"2024-05-09T07:39:17.746477+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:39:17.746477+0000","last_undegraded":"2024-05-09T07:39:17.746477+0000","last_fullsized":"2024-05-09T07:39:17.746477+0000","mapping_epoch":17,"log_start":"27'2600","ondisk_log_start":"27'2600","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7940,"log_dups_size":0,"ondisk_log_size":7940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":627,"num_read_kb":4520,"num_write":10533,"num_write_kb":76795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9186","reported_seq":9429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.849229+0000","last_change":"2024-05-09T07:14:26.592176+0000","last_active":"2024-05-09T07:39:33.849229+0000","last_peered":"2024-05-09T07:39:33.849229+0000","last_clean":"2024-05-09T07:39:33.849229+0000","last_became_active":"2024-05-09T07:14:26.590721+0000","last_became_peered":"2024-05-09T07:14:26.590721+0000","last_unstale":"2024-05-09T07:39:33.849229+0000","last_undegraded":"2024-05-09T07:39:33.849229+0000","last_fullsized":"2024-05-09T07:39:33.849229+0000","mapping_epoch":26,"log_start":"27'1828","ondisk_log_start":"27'1828","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7358,"log_dups_size":0,"ondisk_log_size":7358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:38.451332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4278543,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":221,"num_read_kb":56134,"num_write":6500,"num_write_kb":978701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"27'2569","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.706846+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:39:33.706846+0000","last_peered":"2024-05-09T07:39:33.706846+0000","last_clean":"2024-05-09T07:39:33.706846+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:39:33.706846+0000","last_undegraded":"2024-05-09T07:39:33.706846+0000","last_fullsized":"2024-05-09T07:39:33.706846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6174","reported_seq":6212,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:27.333611+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:39:27.333611+0000","last_peered":"2024-05-09T07:39:27.333611+0000","last_clean":"2024-05-09T07:39:27.333611+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:39:27.333611+0000","last_undegraded":"2024-05-09T07:39:27.333611+0000","last_fullsized":"2024-05-09T07:39:27.333611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6174,"log_dups_size":0,"ondisk_log_size":6174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6180,"num_write_kb":41394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9167","reported_seq":9510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.231279+0000","last_change":"2024-05-09T07:14:26.596397+0000","last_active":"2024-05-09T07:39:34.231279+0000","last_peered":"2024-05-09T07:39:34.231279+0000","last_clean":"2024-05-09T07:39:34.231279+0000","last_became_active":"2024-05-09T07:14:26.595936+0000","last_became_peered":"2024-05-09T07:14:26.595936+0000","last_unstale":"2024-05-09T07:39:34.231279+0000","last_undegraded":"2024-05-09T07:39:34.231279+0000","last_fullsized":"2024-05-09T07:39:34.231279+0000","mapping_epoch":26,"log_start":"27'2010","ondisk_log_start":"27'2010","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7157,"log_dups_size":0,"ondisk_log_size":7157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:45.938473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1040384,"num_objects":4,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":305,"num_read_kb":77470,"num_write":6442,"num_write_kb":989255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2602","reported_seq":2637,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.576974+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:39:34.576974+0000","last_peered":"2024-05-09T07:39:34.576974+0000","last_clean":"2024-05-09T07:39:34.576974+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:39:34.576974+0000","last_undegraded":"2024-05-09T07:39:34.576974+0000","last_fullsized":"2024-05-09T07:39:34.576974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2602,"log_dups_size":0,"ondisk_log_size":2602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5738","reported_seq":5776,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:06.691749+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:39:06.691749+0000","last_peered":"2024-05-09T07:39:06.691749+0000","last_clean":"2024-05-09T07:39:06.691749+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:39:06.691749+0000","last_undegraded":"2024-05-09T07:39:06.691749+0000","last_fullsized":"2024-05-09T07:39:06.691749+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5738,"log_dups_size":0,"ondisk_log_size":5738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5739,"num_write_kb":52939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9112","reported_seq":9479,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.730627+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:39:33.730627+0000","last_peered":"2024-05-09T07:39:33.730627+0000","last_clean":"2024-05-09T07:39:33.730627+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:39:33.730627+0000","last_undegraded":"2024-05-09T07:39:33.730627+0000","last_fullsized":"2024-05-09T07:39:33.730627+0000","mapping_epoch":22,"log_start":"27'2805","ondisk_log_start":"27'2805","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6307,"log_dups_size":0,"ondisk_log_size":6307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1304576,"num_objects":6,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":335,"num_read_kb":85090,"num_write":6440,"num_write_kb":967374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2569","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.769929+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:39:33.769929+0000","last_peered":"2024-05-09T07:39:33.769929+0000","last_clean":"2024-05-09T07:39:33.769929+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:39:33.769929+0000","last_undegraded":"2024-05-09T07:39:33.769929+0000","last_fullsized":"2024-05-09T07:39:33.769929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2569,"log_dups_size":0,"ondisk_log_size":2569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'5942","reported_seq":6846,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:15.634681+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:38:15.634681+0000","last_peered":"2024-05-09T07:38:15.634681+0000","last_clean":"2024-05-09T07:38:15.634681+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:38:15.634681+0000","last_undegraded":"2024-05-09T07:38:15.634681+0000","last_fullsized":"2024-05-09T07:38:15.634681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5942,"log_dups_size":0,"ondisk_log_size":5942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":867,"num_read_kb":4659,"num_write":5943,"num_write_kb":49048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9593","reported_seq":9981,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.605018+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:39:33.605018+0000","last_peered":"2024-05-09T07:39:33.605018+0000","last_clean":"2024-05-09T07:39:33.605018+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:39:33.605018+0000","last_undegraded":"2024-05-09T07:39:33.605018+0000","last_fullsized":"2024-05-09T07:39:33.605018+0000","mapping_epoch":22,"log_start":"27'3323","ondisk_log_start":"27'3323","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6270,"log_dups_size":0,"ondisk_log_size":6270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746850,"num_objects":10,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6812,"num_write_kb":1059940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9162","reported_seq":9389,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.603712+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:39:34.603712+0000","last_peered":"2024-05-09T07:39:34.603712+0000","last_clean":"2024-05-09T07:39:34.603712+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:39:34.603712+0000","last_undegraded":"2024-05-09T07:39:34.603712+0000","last_fullsized":"2024-05-09T07:39:34.603712+0000","mapping_epoch":22,"log_start":"27'1813","ondisk_log_start":"27'1813","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7349,"log_dups_size":0,"ondisk_log_size":7349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":49276,"num_write":6559,"num_write_kb":912347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2528","reported_seq":2564,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.594958+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:39:34.594958+0000","last_peered":"2024-05-09T07:39:34.594958+0000","last_clean":"2024-05-09T07:39:34.594958+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:39:34.594958+0000","last_undegraded":"2024-05-09T07:39:34.594958+0000","last_fullsized":"2024-05-09T07:39:34.594958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9067","reported_seq":9105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:08.929933+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:38:08.929933+0000","last_peered":"2024-05-09T07:38:08.929933+0000","last_clean":"2024-05-09T07:38:08.929933+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:38:08.929933+0000","last_undegraded":"2024-05-09T07:38:08.929933+0000","last_fullsized":"2024-05-09T07:38:08.929933+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9064,"num_write_kb":67566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8837","reported_seq":10023,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.965223+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:39:33.965223+0000","last_peered":"2024-05-09T07:39:33.965223+0000","last_clean":"2024-05-09T07:39:33.965223+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:39:33.965223+0000","last_undegraded":"2024-05-09T07:39:33.965223+0000","last_fullsized":"2024-05-09T07:39:33.965223+0000","mapping_epoch":22,"log_start":"27'900","ondisk_log_start":"27'900","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7937,"log_dups_size":0,"ondisk_log_size":7937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1948104,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1153,"num_read_kb":292862,"num_write":6075,"num_write_kb":877058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2618","reported_seq":2654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.701610+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:39:33.701610+0000","last_peered":"2024-05-09T07:39:33.701610+0000","last_clean":"2024-05-09T07:39:33.701610+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:39:33.701610+0000","last_undegraded":"2024-05-09T07:39:33.701610+0000","last_fullsized":"2024-05-09T07:39:33.701610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2618,"log_dups_size":0,"ondisk_log_size":2618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5353","reported_seq":5391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:13.460729+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:38:13.460729+0000","last_peered":"2024-05-09T07:38:13.460729+0000","last_clean":"2024-05-09T07:38:13.460729+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:38:13.460729+0000","last_undegraded":"2024-05-09T07:38:13.460729+0000","last_fullsized":"2024-05-09T07:38:13.460729+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5353,"log_dups_size":0,"ondisk_log_size":5353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5353,"num_write_kb":44481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9693","reported_seq":9731,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:43.717707+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:38:43.717707+0000","last_peered":"2024-05-09T07:38:43.717707+0000","last_clean":"2024-05-09T07:38:43.717707+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:38:43.717707+0000","last_undegraded":"2024-05-09T07:38:43.717707+0000","last_fullsized":"2024-05-09T07:38:43.717707+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7993,"log_dups_size":0,"ondisk_log_size":7993,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9690,"num_write_kb":72019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2555","reported_seq":2590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.417888+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:39:34.417888+0000","last_peered":"2024-05-09T07:39:34.417888+0000","last_clean":"2024-05-09T07:39:34.417888+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:39:34.417888+0000","last_undegraded":"2024-05-09T07:39:34.417888+0000","last_fullsized":"2024-05-09T07:39:34.417888+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2555,"log_dups_size":0,"ondisk_log_size":2555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9392","reported_seq":9447,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.677629+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:39:33.677629+0000","last_peered":"2024-05-09T07:39:33.677629+0000","last_clean":"2024-05-09T07:39:33.677629+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:39:33.677629+0000","last_undegraded":"2024-05-09T07:39:33.677629+0000","last_fullsized":"2024-05-09T07:39:33.677629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9392,"log_dups_size":0,"ondisk_log_size":9392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1565758,"num_objects":9,"num_object_clones":0,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":24,"num_read_kb":6096,"num_write":6594,"num_write_kb":983207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8024","reported_seq":8062,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:48.275994+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:38:48.275994+0000","last_peered":"2024-05-09T07:38:48.275994+0000","last_clean":"2024-05-09T07:38:48.275994+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:38:48.275994+0000","last_undegraded":"2024-05-09T07:38:48.275994+0000","last_fullsized":"2024-05-09T07:38:48.275994+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6324,"log_dups_size":0,"ondisk_log_size":6324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8026,"num_write_kb":66635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2597","reported_seq":2632,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.242028+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:39:34.242028+0000","last_peered":"2024-05-09T07:39:34.242028+0000","last_clean":"2024-05-09T07:39:34.242028+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:39:34.242028+0000","last_undegraded":"2024-05-09T07:39:34.242028+0000","last_fullsized":"2024-05-09T07:39:34.242028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2597,"log_dups_size":0,"ondisk_log_size":2597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9823","reported_seq":9975,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.979566+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:39:33.979566+0000","last_peered":"2024-05-09T07:39:33.979566+0000","last_clean":"2024-05-09T07:39:33.979566+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:39:33.979566+0000","last_undegraded":"2024-05-09T07:39:33.979566+0000","last_fullsized":"2024-05-09T07:39:33.979566+0000","mapping_epoch":22,"log_start":"27'3543","ondisk_log_start":"27'3543","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1081890,"num_objects":7,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":121,"num_read_kb":30734,"num_write":7010,"num_write_kb":1030226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2491","reported_seq":2529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.847333+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:39:33.847333+0000","last_peered":"2024-05-09T07:39:33.847333+0000","last_clean":"2024-05-09T07:39:33.847333+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:39:33.847333+0000","last_undegraded":"2024-05-09T07:39:33.847333+0000","last_fullsized":"2024-05-09T07:39:33.847333+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2491,"log_dups_size":0,"ondisk_log_size":2491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2611","reported_seq":2648,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.123638+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:39:33.123638+0000","last_peered":"2024-05-09T07:39:33.123638+0000","last_clean":"2024-05-09T07:39:33.123638+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:39:33.123638+0000","last_undegraded":"2024-05-09T07:39:33.123638+0000","last_fullsized":"2024-05-09T07:39:33.123638+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2515","reported_seq":2553,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.622088+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:39:33.622088+0000","last_peered":"2024-05-09T07:39:33.622088+0000","last_clean":"2024-05-09T07:39:33.622088+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:39:33.622088+0000","last_undegraded":"2024-05-09T07:39:33.622088+0000","last_fullsized":"2024-05-09T07:39:33.622088+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2515,"log_dups_size":0,"ondisk_log_size":2515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2604","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.371669+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:39:34.371669+0000","last_peered":"2024-05-09T07:39:34.371669+0000","last_clean":"2024-05-09T07:39:34.371669+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:39:34.371669+0000","last_undegraded":"2024-05-09T07:39:34.371669+0000","last_fullsized":"2024-05-09T07:39:34.371669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2604,"log_dups_size":0,"ondisk_log_size":2604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2596","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.755716+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:39:33.755716+0000","last_peered":"2024-05-09T07:39:33.755716+0000","last_clean":"2024-05-09T07:39:33.755716+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:39:33.755716+0000","last_undegraded":"2024-05-09T07:39:33.755716+0000","last_fullsized":"2024-05-09T07:39:33.755716+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2596,"log_dups_size":0,"ondisk_log_size":2596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2482","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.054999+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:39:34.054999+0000","last_peered":"2024-05-09T07:39:34.054999+0000","last_clean":"2024-05-09T07:39:34.054999+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:39:34.054999+0000","last_undegraded":"2024-05-09T07:39:34.054999+0000","last_fullsized":"2024-05-09T07:39:34.054999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2610","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.002968+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:39:34.002968+0000","last_peered":"2024-05-09T07:39:34.002968+0000","last_clean":"2024-05-09T07:39:34.002968+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:39:34.002968+0000","last_undegraded":"2024-05-09T07:39:34.002968+0000","last_fullsized":"2024-05-09T07:39:34.002968+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2610,"log_dups_size":0,"ondisk_log_size":2610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2582","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.942776+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:39:33.942776+0000","last_peered":"2024-05-09T07:39:33.942776+0000","last_clean":"2024-05-09T07:39:33.942776+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:39:33.942776+0000","last_undegraded":"2024-05-09T07:39:33.942776+0000","last_fullsized":"2024-05-09T07:39:33.942776+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2582,"log_dups_size":0,"ondisk_log_size":2582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2566","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.702866+0000","last_change":"2024-05-09T07:14:26.646404+0000","last_active":"2024-05-09T07:39:33.702866+0000","last_peered":"2024-05-09T07:39:33.702866+0000","last_clean":"2024-05-09T07:39:33.702866+0000","last_became_active":"2024-05-09T07:14:26.646058+0000","last_became_peered":"2024-05-09T07:14:26.646058+0000","last_unstale":"2024-05-09T07:39:33.702866+0000","last_undegraded":"2024-05-09T07:39:33.702866+0000","last_fullsized":"2024-05-09T07:39:33.702866+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2566,"log_dups_size":0,"ondisk_log_size":2566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'2525","reported_seq":2562,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.518809+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:39:33.518809+0000","last_peered":"2024-05-09T07:39:33.518809+0000","last_clean":"2024-05-09T07:39:33.518809+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:39:33.518809+0000","last_undegraded":"2024-05-09T07:39:33.518809+0000","last_fullsized":"2024-05-09T07:39:33.518809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2525,"log_dups_size":0,"ondisk_log_size":2525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2599","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.840412+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:39:33.840412+0000","last_peered":"2024-05-09T07:39:33.840412+0000","last_clean":"2024-05-09T07:39:33.840412+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:39:33.840412+0000","last_undegraded":"2024-05-09T07:39:33.840412+0000","last_fullsized":"2024-05-09T07:39:33.840412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2608","reported_seq":2644,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.225626+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:39:34.225626+0000","last_peered":"2024-05-09T07:39:34.225626+0000","last_clean":"2024-05-09T07:39:34.225626+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:39:34.225626+0000","last_undegraded":"2024-05-09T07:39:34.225626+0000","last_fullsized":"2024-05-09T07:39:34.225626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2608,"log_dups_size":0,"ondisk_log_size":2608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2601","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:32.409252+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:39:32.409252+0000","last_peered":"2024-05-09T07:39:32.409252+0000","last_clean":"2024-05-09T07:39:32.409252+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:39:32.409252+0000","last_undegraded":"2024-05-09T07:39:32.409252+0000","last_fullsized":"2024-05-09T07:39:32.409252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2606","reported_seq":2642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.135665+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:39:34.135665+0000","last_peered":"2024-05-09T07:39:34.135665+0000","last_clean":"2024-05-09T07:39:34.135665+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:39:34.135665+0000","last_undegraded":"2024-05-09T07:39:34.135665+0000","last_fullsized":"2024-05-09T07:39:34.135665+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2544","reported_seq":2580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.079417+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:39:33.079417+0000","last_peered":"2024-05-09T07:39:33.079417+0000","last_clean":"2024-05-09T07:39:33.079417+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:39:33.079417+0000","last_undegraded":"2024-05-09T07:39:33.079417+0000","last_fullsized":"2024-05-09T07:39:33.079417+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2544,"log_dups_size":0,"ondisk_log_size":2544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2609","reported_seq":2646,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.792677+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:39:33.792677+0000","last_peered":"2024-05-09T07:39:33.792677+0000","last_clean":"2024-05-09T07:39:33.792677+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:39:33.792677+0000","last_undegraded":"2024-05-09T07:39:33.792677+0000","last_fullsized":"2024-05-09T07:39:33.792677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2609,"log_dups_size":0,"ondisk_log_size":2609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2547","reported_seq":2584,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.736964+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:39:33.736964+0000","last_peered":"2024-05-09T07:39:33.736964+0000","last_clean":"2024-05-09T07:39:33.736964+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:39:33.736964+0000","last_undegraded":"2024-05-09T07:39:33.736964+0000","last_fullsized":"2024-05-09T07:39:33.736964+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2547,"log_dups_size":0,"ondisk_log_size":2547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2669","reported_seq":2706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.483917+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:39:34.483917+0000","last_peered":"2024-05-09T07:39:34.483917+0000","last_clean":"2024-05-09T07:39:34.483917+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:39:34.483917+0000","last_undegraded":"2024-05-09T07:39:34.483917+0000","last_fullsized":"2024-05-09T07:39:34.483917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2669,"log_dups_size":0,"ondisk_log_size":2669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2520","reported_seq":2556,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.617853+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:39:34.617853+0000","last_peered":"2024-05-09T07:39:34.617853+0000","last_clean":"2024-05-09T07:39:34.617853+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:39:34.617853+0000","last_undegraded":"2024-05-09T07:39:34.617853+0000","last_fullsized":"2024-05-09T07:39:34.617853+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2504","reported_seq":2540,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.803579+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:39:33.803579+0000","last_peered":"2024-05-09T07:39:33.803579+0000","last_clean":"2024-05-09T07:39:33.803579+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:39:33.803579+0000","last_undegraded":"2024-05-09T07:39:33.803579+0000","last_fullsized":"2024-05-09T07:39:33.803579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2504,"log_dups_size":0,"ondisk_log_size":2504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2482","reported_seq":2517,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.172679+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:39:34.172679+0000","last_peered":"2024-05-09T07:39:34.172679+0000","last_clean":"2024-05-09T07:39:34.172679+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:39:34.172679+0000","last_undegraded":"2024-05-09T07:39:34.172679+0000","last_fullsized":"2024-05-09T07:39:34.172679+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2503","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.838109+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:39:33.838109+0000","last_peered":"2024-05-09T07:39:33.838109+0000","last_clean":"2024-05-09T07:39:33.838109+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:39:33.838109+0000","last_undegraded":"2024-05-09T07:39:33.838109+0000","last_fullsized":"2024-05-09T07:39:33.838109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2632","reported_seq":2668,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.753412+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:39:33.753412+0000","last_peered":"2024-05-09T07:39:33.753412+0000","last_clean":"2024-05-09T07:39:33.753412+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:39:33.753412+0000","last_undegraded":"2024-05-09T07:39:33.753412+0000","last_fullsized":"2024-05-09T07:39:33.753412+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2632,"log_dups_size":0,"ondisk_log_size":2632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2599","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.441391+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:39:34.441391+0000","last_peered":"2024-05-09T07:39:34.441391+0000","last_clean":"2024-05-09T07:39:34.441391+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:39:34.441391+0000","last_undegraded":"2024-05-09T07:39:34.441391+0000","last_fullsized":"2024-05-09T07:39:34.441391+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2599,"log_dups_size":0,"ondisk_log_size":2599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2537","reported_seq":2573,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.832419+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:39:33.832419+0000","last_peered":"2024-05-09T07:39:33.832419+0000","last_clean":"2024-05-09T07:39:33.832419+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:39:33.832419+0000","last_undegraded":"2024-05-09T07:39:33.832419+0000","last_fullsized":"2024-05-09T07:39:33.832419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2537,"log_dups_size":0,"ondisk_log_size":2537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.487057+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:39:34.487057+0000","last_peered":"2024-05-09T07:39:34.487057+0000","last_clean":"2024-05-09T07:39:34.487057+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:39:34.487057+0000","last_undegraded":"2024-05-09T07:39:34.487057+0000","last_fullsized":"2024-05-09T07:39:34.487057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2503","reported_seq":2539,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.726849+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:39:33.726849+0000","last_peered":"2024-05-09T07:39:33.726849+0000","last_clean":"2024-05-09T07:39:33.726849+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:39:33.726849+0000","last_undegraded":"2024-05-09T07:39:33.726849+0000","last_fullsized":"2024-05-09T07:39:33.726849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2503,"log_dups_size":0,"ondisk_log_size":2503,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2542","reported_seq":2577,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.644003+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:39:34.644003+0000","last_peered":"2024-05-09T07:39:34.644003+0000","last_clean":"2024-05-09T07:39:34.644003+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:39:34.644003+0000","last_undegraded":"2024-05-09T07:39:34.644003+0000","last_fullsized":"2024-05-09T07:39:34.644003+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2580","reported_seq":2618,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:33.459093+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:39:33.459093+0000","last_peered":"2024-05-09T07:39:33.459093+0000","last_clean":"2024-05-09T07:39:33.459093+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:39:33.459093+0000","last_undegraded":"2024-05-09T07:39:33.459093+0000","last_fullsized":"2024-05-09T07:39:33.459093+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2580,"log_dups_size":0,"ondisk_log_size":2580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2658","reported_seq":2695,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.225175+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:39:34.225175+0000","last_peered":"2024-05-09T07:39:34.225175+0000","last_clean":"2024-05-09T07:39:34.225175+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:39:34.225175+0000","last_undegraded":"2024-05-09T07:39:34.225175+0000","last_fullsized":"2024-05-09T07:39:34.225175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2658,"log_dups_size":0,"ondisk_log_size":2658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2496","reported_seq":2534,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.141290+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:39:34.141290+0000","last_peered":"2024-05-09T07:39:34.141290+0000","last_clean":"2024-05-09T07:39:34.141290+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:39:34.141290+0000","last_undegraded":"2024-05-09T07:39:34.141290+0000","last_fullsized":"2024-05-09T07:39:34.141290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2496,"log_dups_size":0,"ondisk_log_size":2496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2482","reported_seq":2519,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.536252+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:39:34.536252+0000","last_peered":"2024-05-09T07:39:34.536252+0000","last_clean":"2024-05-09T07:39:34.536252+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:39:34.536252+0000","last_undegraded":"2024-05-09T07:39:34.536252+0000","last_fullsized":"2024-05-09T07:39:34.536252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2482,"log_dups_size":0,"ondisk_log_size":2482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":61502438,"num_objects":168,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":12270,"num_read_kb":3117350,"num_write":214562,"num_write_kb":30722105,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":120573952,"data_stored":120573952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232059,"ondisk_log_size":232059,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1007,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164175,"ondisk_log_size":164175,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":248505113,"num_objects":195,"num_object_clones":0,"num_object_copies":390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":195,"num_whiteouts":0,"num_read":3177,"num_read_kb":19424,"num_write":211653,"num_write_kb":1695367,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":127,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497180672,"data_stored":497116653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192901,"ondisk_log_size":192901,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575163,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":970376,"kb_used_data":64640,"kb_used_omap":1,"kb_used_meta":905726,"kb_avail":93401464,"statfs":{"total":96636764160,"available":95643099136,"internally_reserved":0,"allocated":66191360,"data_stored":66012455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":927463882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575162,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1129536,"kb_used_data":91960,"kb_used_omap":1,"kb_used_meta":1037566,"kb_avail":93242304,"statfs":{"total":96636764160,"available":95480119296,"internally_reserved":0,"allocated":94167040,"data_stored":93987681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1062468038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575162,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1019696,"kb_used_data":81896,"kb_used_omap":1,"kb_used_meta":937662,"kb_avail":93352144,"statfs":{"total":96636764160,"available":95592595456,"internally_reserved":0,"allocated":83861504,"data_stored":83687882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960166341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575161,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062008,"kb_used_data":84016,"kb_used_omap":1,"kb_used_meta":977854,"kb_avail":93309832,"statfs":{"total":96636764160,"available":95549267968,"internally_reserved":0,"allocated":86032384,"data_stored":85850833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001322950},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575163,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1279116,"kb_used_data":128116,"kb_used_omap":1,"kb_used_meta":1150078,"kb_avail":93092724,"statfs":{"total":96636764160,"available":95326949376,"internally_reserved":0,"allocated":131190784,"data_stored":131013875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1177680327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575164,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729932,"kb_used_data":76932,"kb_used_omap":1,"kb_used_meta":652990,"kb_avail":93641908,"statfs":{"total":96636764160,"available":95889313792,"internally_reserved":0,"allocated":78778368,"data_stored":78606656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668662213},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575165,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1251056,"kb_used_data":82072,"kb_used_omap":1,"kb_used_meta":1168062,"kb_avail":93120784,"statfs":{"total":96636764160,"available":95355682816,"internally_reserved":0,"allocated":84041728,"data_stored":83864307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1196095941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":898152,"kb_used_data":68896,"kb_used_omap":1,"kb_used_meta":829246,"kb_avail":93473688,"statfs":{"total":96636764160,"available":95717056512,"internally_reserved":0,"allocated":70549504,"data_stored":70364913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":849148357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,5,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67109502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54583296,"data_stored":54582378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71409664,"data_stored":71398347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46141440,"data_stored":46137434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":68136960,"data_stored":68126990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":55549952,"data_stored":55543988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1048576,"data_stored":1048576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2760704,"data_stored":2760704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":23805952,"data_stored":23805952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":29478912,"data_stored":29478912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16297984,"data_stored":16297984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":21856256,"data_stored":21856256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24752128,"data_stored":24752128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":573440,"data_stored":573440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:39:39.146 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:39:39.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.273+0000 7f2208f70640 1 Processor -- start 2024-05-09T07:39:39.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.273+0000 7f2208f70640 1 -- start start 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2208f70640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f22040c5610 0x7f22040c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2208f70640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f22040be320 0x7f2204155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2208f70640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2204159840 con 0x7f22040c8b90 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2208f70640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f2204159310 con 0x7f22040c5610 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2208f70640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f2204159560 con 0x7f22040be320 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f22040c5610 0x7f22040c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f22040c5610 0x7f22040c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34722/0 (socket says 172.21.15.160:34722) 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2202d9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f22040c8b90 0x7f2204155850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34606/0 (socket says 172.21.15.160:34606) 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 -- 172.21.15.160:0/1085912050 learned_addr learned my addr 172.21.15.160:0/1085912050 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:39.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2201d9b640 1 --2- 172.21.15.160:0/1085912050 >> v2:172.21.15.188:3300/0 conn(0x7f22040be320 0x7f2204155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 -- 172.21.15.160:0/1085912050 >> v2:172.21.15.188:3300/0 conn(0x7f22040be320 msgr2=0x7f2204155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 --2- 172.21.15.160:0/1085912050 >> v2:172.21.15.188:3300/0 conn(0x7f22040be320 0x7f2204155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 -- 172.21.15.160:0/1085912050 >> v1:172.21.15.160:6789/0 conn(0x7f22040c8b90 legacy=0x7f2204155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220259c640 1 -- 172.21.15.160:0/1085912050 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040c77d0 con 0x7f22040c5610 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f220159a640 1 -- 172.21.15.160:0/1085912050 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3493541282 0 0) 0x7f2204159840 con 0x7f22040c8b90 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.274+0000 7f2201d9b640 1 --2- 172.21.15.160:0/1085912050 >> v2:172.21.15.188:3300/0 conn(0x7f22040be320 0x7f2204155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:39.287 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.275+0000 7f220259c640 1 --2- 172.21.15.160:0/1085912050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c5610 0x7f22040c59f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f220406d860 tx=0x7f21f402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650761727b017279 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:39.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f220159a640 1 -- 172.21.15.160:0/1085912050 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4037070 con 0x7f22040c5610 2024-05-09T07:39:39.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f220159a640 1 -- 172.21.15.160:0/1085912050 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21f4032440 con 0x7f22040c5610 2024-05-09T07:39:39.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f220159a640 1 -- 172.21.15.160:0/1085912050 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21f4032810 con 0x7f22040c5610 2024-05-09T07:39:39.290 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 -- 172.21.15.160:0/1085912050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c5610 msgr2=0x7f22040c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 --2- 172.21.15.160:0/1085912050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c5610 0x7f22040c59f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f220406d860 tx=0x7f21f402a150 comp rx=0 tx=0).stop 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 -- 172.21.15.160:0/1085912050 shutdown_connections 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 --2- 172.21.15.160:0/1085912050 >> v2:172.21.15.188:3300/0 conn(0x7f22040be320 0x7f2204155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 --2- 172.21.15.160:0/1085912050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c5610 0x7f22040c59f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 -- 172.21.15.160:0/1085912050 >> 172.21.15.160:0/1085912050 conn(0x7f2204072e90 msgr2=0x7f22040ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 -- 172.21.15.160:0/1085912050 shutdown_connections 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.276+0000 7f2208f70640 1 -- 172.21.15.160:0/1085912050 wait complete. 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 Processor -- start 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 -- start start 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22040be320 0x7f2204154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040c5610 0x7f220414bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 0x7f220414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:39.291 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f22040ad6a0 con 0x7f22040c8b90 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f22040c7d90 con 0x7f22040c5610 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2208f70640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f2204155f60 con 0x7f22040be320 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2202d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 0x7f220414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2202d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 0x7f220414c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34726/0 (socket says 172.21.15.160:34726) 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.278+0000 7f2202d9d640 1 -- 172.21.15.160:0/2370951856 learned_addr learned my addr 172.21.15.160:0/2370951856 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2201d9b640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040c5610 0x7f220414bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2202d9d640 1 -- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22040be320 msgr2=0x7f2204154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2202d9d640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22040be320 0x7f2204154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2202d9d640 1 -- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040c5610 msgr2=0x7f220414bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2202d9d640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040c5610 0x7f220414bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2202d9d640 1 -- 172.21.15.160:0/2370951856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22040acd40 con 0x7f22040c8b90 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2201d9b640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040c5610 0x7f220414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2202d9d640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 0x7f220414c480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c700 tx=0x7f21fc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:39.292 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc018070 con 0x7f22040c8b90 2024-05-09T07:39:39.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f21fc00ce10 con 0x7f22040c8b90 2024-05-09T07:39:39.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22041eb7f0 con 0x7f22040c8b90 2024-05-09T07:39:39.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21fc00a860 con 0x7f22040c8b90 2024-05-09T07:39:39.293 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.279+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22040abd60 con 0x7f22040c8b90 2024-05-09T07:39:39.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.280+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22040abfa0 con 0x7f22040c8b90 2024-05-09T07:39:39.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.281+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f21fc01b070 con 0x7f22040c8b90 2024-05-09T07:39:39.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.282+0000 7f21faffd640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f21e406be60 0x7f21e406e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:39.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.282+0000 7f220259c640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f21e406be60 0x7f21e406e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:39.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.282+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f21fc08a7a0 con 0x7f22040c8b90 2024-05-09T07:39:39.294 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.282+0000 7f220259c640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f21e406be60 0x7f21e406e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21f4034000 tx=0x7f21f402c000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:39.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.286+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22040abfa0 con 0x7f22040c8b90 2024-05-09T07:39:39.481 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.470+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f22040756c0 con 0x7f22040c8b90 2024-05-09T07:39:39.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.472+0000 7f21faffd640 1 -- 172.21.15.160:0/2370951856 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7f21fc058d40 con 0x7f22040c8b90 2024-05-09T07:39:39.483 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:39:39.483 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":27,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:14:26.582005+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f21e406be60 msgr2=0x7f21e406e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f21e406be60 0x7f21e406e320 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f21f4034000 tx=0x7f21f402c000 comp rx=0 tx=0).stop 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 msgr2=0x7f220414c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 0x7f220414c480 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f21fc00c700 tx=0x7f21fc00cbc0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 shutdown_connections 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f21e406be60 0x7f21e406e320 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f22040c5610 0x7f220414bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f22040c8b90 0x7f220414c480 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 --2- 172.21.15.160:0/2370951856 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f22040be320 0x7f2204154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:39.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 >> 172.21.15.160:0/2370951856 conn(0x7f2204072e90 msgr2=0x7f2204158200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:39.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 shutdown_connections 2024-05-09T07:39:39.486 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:39.474+0000 7f2208f70640 1 -- 172.21.15.160:0/2370951856 wait complete. 2024-05-09T07:39:40.497 INFO:tasks.ceph:Scrubbing osd.0 2024-05-09T07:39:40.497 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:40.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.594+0000 7f23c5961640 1 Processor -- start 2024-05-09T07:39:40.605 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23c5961640 1 -- start start 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23c5961640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c0070e20 0x7f23c00c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23c5961640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f23c00c8aa0 0x7f23c00c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23c5961640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23c0070690 con 0x7f23c0152a30 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23c5961640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f23c00a7d40 con 0x7f23c00c8aa0 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23c5961640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f23c00a8020 con 0x7f23c0070e20 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23beffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f23c0152a30 0x7f23c00c2a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34612/0 (socket says 172.21.15.160:34612) 2024-05-09T07:39:40.607 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23beffd640 1 -- 172.21.15.160:0/3875044977 learned_addr learned my addr 172.21.15.160:0/3875044977 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.595+0000 7f23bf7fe640 1 --2- 172.21.15.160:0/3875044977 >> v2:172.21.15.160:3300/0 conn(0x7f23c00c8aa0 0x7f23c00c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23be7fc640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c0070e20 0x7f23c00c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23bdffb640 1 -- 172.21.15.160:0/3875044977 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1999206279 0 0) 0x7f23c0070690 con 0x7f23c0152a30 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23bdffb640 1 -- 172.21.15.160:0/3875044977 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f23ac003520 con 0x7f23c0152a30 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23bf7fe640 1 -- 172.21.15.160:0/3875044977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c0070e20 msgr2=0x7f23c00c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23bf7fe640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c0070e20 0x7f23c00c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23bf7fe640 1 -- 172.21.15.160:0/3875044977 >> v1:172.21.15.160:6789/0 conn(0x7f23c0152a30 legacy=0x7f23c00c2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23bf7fe640 1 -- 172.21.15.160:0/3875044977 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c00a8300 con 0x7f23c00c8aa0 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.596+0000 7f23be7fc640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c0070e20 0x7f23c00c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:40.608 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.597+0000 7f23bf7fe640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c00c8aa0 0x7f23c00c8ed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009d90 tx=0x7f23b802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b434579ee249d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.597+0000 7f23bdffb640 1 -- 172.21.15.160:0/3875044977 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007740 con 0x7f23c00c8aa0 2024-05-09T07:39:40.609 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.597+0000 7f23bdffb640 1 -- 172.21.15.160:0/3875044977 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b80078e0 con 0x7f23c00c8aa0 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.597+0000 7f23bdffb640 1 -- 172.21.15.160:0/3875044977 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b8007cb0 con 0x7f23c00c8aa0 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.597+0000 7f23c5961640 1 -- 172.21.15.160:0/3875044977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c00c8aa0 msgr2=0x7f23c00c8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.597+0000 7f23c5961640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c00c8aa0 0x7f23c00c8ed0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f23b8009d90 tx=0x7f23b802a660 comp rx=0 tx=0).stop 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.598+0000 7f23c5961640 1 -- 172.21.15.160:0/3875044977 shutdown_connections 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.598+0000 7f23c5961640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c00c8aa0 0x7f23c00c8ed0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.598+0000 7f23c5961640 1 --2- 172.21.15.160:0/3875044977 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c0070e20 0x7f23c00c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.598+0000 7f23c5961640 1 -- 172.21.15.160:0/3875044977 >> 172.21.15.160:0/3875044977 conn(0x7f23c0072a80 msgr2=0x7f23c0072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.598+0000 7f23c5961640 1 -- 172.21.15.160:0/3875044977 shutdown_connections 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.598+0000 7f23c5961640 1 -- 172.21.15.160:0/3875044977 wait complete. 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.599+0000 7f23c5961640 1 Processor -- start 2024-05-09T07:39:40.610 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.599+0000 7f23c5961640 1 -- start start 2024-05-09T07:39:40.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.599+0000 7f23c5961640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c0070e20 0x7f23c01ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c00c8aa0 0x7f23c0154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23c0152a30 0x7f23c00ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.613 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f23c0152e10 con 0x7f23c0070e20 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f23c00712b0 con 0x7f23c0152a30 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f23c00c6240 con 0x7f23c00c8aa0 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c0070e20 0x7f23c01ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c0070e20 0x7f23c01ef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34758/0 (socket says 172.21.15.160:34758) 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 -- 172.21.15.160:0/3839390691 learned_addr learned my addr 172.21.15.160:0/3839390691 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23be7fc640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c00c8aa0 0x7f23c0154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23bf7fe640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23c0152a30 0x7f23c00ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 -- 172.21.15.160:0/3839390691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c00c8aa0 msgr2=0x7f23c0154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f23c00c8aa0 0x7f23c0154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 -- 172.21.15.160:0/3839390691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23c0152a30 msgr2=0x7f23c00ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f23c0152a30 0x7f23c00ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.614 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 -- 172.21.15.160:0/3839390691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23c01ef590 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23beffd640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c0070e20 0x7f23c01ef050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f23b000c430 tx=0x7f23b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b0017070 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23b0005ce0 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23c01518e0 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23b00072f0 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.600+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23c00acd60 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.601+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23c014b090 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f23b001a070 con 0x7f23c0070e20 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23b7fff640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f23a006be30 0x7f23a006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.615 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f23b008a6c0 con 0x7f23c0070e20 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23be7fc640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f23a006be30 0x7f23a006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23b7fff640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f23a00757c0 0x7f23a0077c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 --> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23b0015e70 con 0x7f23a00757c0 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.602+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23b0091050 con 0x7f23c0070e20 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.603+0000 7f23bf7fe640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f23a00757c0 0x7f23a0077c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.603+0000 7f23be7fc640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f23a006be30 0x7f23a006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23c0073170 tx=0x7f23a8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.603+0000 7f23bf7fe640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f23a00757c0 0x7f23a0077c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.616 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.604+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== osd.0 v2:172.21.15.160:6800/3071505742 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23b0015e70 con 0x7f23a00757c0 2024-05-09T07:39:40.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.617+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 --> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f23c0071430 con 0x7f23a00757c0 2024-05-09T07:39:40.628 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.617+0000 7f23b7fff640 1 -- 172.21.15.160:0/3839390691 <== osd.0 v2:172.21.15.160:6800/3071505742 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f23c0071430 con 0x7f23a00757c0 2024-05-09T07:39:40.628 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:40.628 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:40.628 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:40.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f23a00757c0 msgr2=0x7f23a0077c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6800/3071505742,v1:172.21.15.160:6801/3071505742] conn(0x7f23a00757c0 0x7f23a0077c60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f23a006be30 msgr2=0x7f23a006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f23a006be30 0x7f23a006e2f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f23c0073170 tx=0x7f23a8008040 comp rx=0 tx=0).stop 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c0070e20 msgr2=0x7f23c01ef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 --2- 172.21.15.160:0/3839390691 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f23c0070e20 0x7f23c01ef050 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f23b000c430 tx=0x7f23b000c8f0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23bf7fe640 1 -- 172.21.15.160:0/3839390691 reap_dead start 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 shutdown_connections 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 >> 172.21.15.160:0/3839390691 conn(0x7f23c0072a80 msgr2=0x7f23c0155480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 shutdown_connections 2024-05-09T07:39:40.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.618+0000 7f23c5961640 1 -- 172.21.15.160:0/3839390691 wait complete. 2024-05-09T07:39:40.640 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-09T07:39:40.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.732+0000 7f50c27ce640 1 Processor -- start 2024-05-09T07:39:40.743 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.732+0000 7f50c27ce640 1 -- start start 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c27ce640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f50bc0c5610 0x7f50bc0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c27ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c27ce640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50bc159840 con 0x7f50bc0be320 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c27ce640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f50bc159310 con 0x7f50bc0c5610 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c27ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50bc159560 con 0x7f50bc0c8b90 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f50bc0c5610 0x7f50bc0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f50bc0c5610 0x7f50bc0c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34764/0 (socket says 172.21.15.160:34764) 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 -- 172.21.15.160:0/4152696665 learned_addr learned my addr 172.21.15.160:0/4152696665 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c0d6b640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.745 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 -- 172.21.15.160:0/4152696665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 msgr2=0x7f50bc155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 -- 172.21.15.160:0/4152696665 >> v1:172.21.15.188:6789/0 conn(0x7f50bc0be320 legacy=0x7f50bc155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50bbfff640 1 -- 172.21.15.160:0/4152696665 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc0c77d0 con 0x7f50bc0c5610 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.733+0000 7f50c0d6b640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.734+0000 7f50bbfff640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc0c59f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009870 tx=0x7f50a402a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b9a9769565f902 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.734+0000 7f50baffd640 1 -- 172.21.15.160:0/4152696665 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4037070 con 0x7f50bc0c5610 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.734+0000 7f50baffd640 1 -- 172.21.15.160:0/4152696665 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50a4032440 con 0x7f50bc0c5610 2024-05-09T07:39:40.746 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.734+0000 7f50baffd640 1 -- 172.21.15.160:0/4152696665 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a4032810 con 0x7f50bc0c5610 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 -- 172.21.15.160:0/4152696665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 msgr2=0x7f50bc0c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc0c59f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009870 tx=0x7f50a402a130 comp rx=0 tx=0).stop 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 -- 172.21.15.160:0/4152696665 shutdown_connections 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 --2- 172.21.15.160:0/4152696665 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc0c59f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 -- 172.21.15.160:0/4152696665 >> 172.21.15.160:0/4152696665 conn(0x7f50bc072e90 msgr2=0x7f50bc0ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 -- 172.21.15.160:0/4152696665 shutdown_connections 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.735+0000 7f50c27ce640 1 -- 172.21.15.160:0/4152696665 wait complete. 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 Processor -- start 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 -- start start 2024-05-09T07:39:40.750 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50bc0be320 0x7f50bc1ebc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc0c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0ad6a0 con 0x7f50bc0c5610 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f50bc0c7d90 con 0x7f50bc0be320 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.736+0000 7f50c27ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f50bc155f60 con 0x7f50bc0c8b90 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc154440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34780/0 (socket says 172.21.15.160:34780) 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 -- 172.21.15.160:0/3201301958 learned_addr learned my addr 172.21.15.160:0/3201301958 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bbfff640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50bc0be320 0x7f50bc1ebc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50c0d6b640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc0c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 -- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 msgr2=0x7f50bc0c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.751 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc0c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 -- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50bc0be320 msgr2=0x7f50bc1ebc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50bc0be320 0x7f50bc1ebc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 -- 172.21.15.160:0/3201301958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50bc0bff20 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50bb7fe640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc154440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c440 tx=0x7f50ac001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac0052d0 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50ac005470 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50bc0c0200 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ac007260 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.737+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50bc154980 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.738+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50bc0756c0 con 0x7f50bc0c5610 2024-05-09T07:39:40.752 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.739+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f50ac007400 con 0x7f50bc0c5610 2024-05-09T07:39:40.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.739+0000 7f50b8ff9640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f509c06be30 0x7f509c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:40.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.739+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f50ac08a630 con 0x7f50bc0c5610 2024-05-09T07:39:40.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.739+0000 7f50bbfff640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f509c06be30 0x7f509c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:40.753 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.740+0000 7f50bbfff640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f509c06be30 0x7f509c06e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50a4034000 tx=0x7f50a402e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:40.754 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.743+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f50ac058630 con 0x7f50bc0c5610 2024-05-09T07:39:40.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.923+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f50bc0be100 con 0x7f509c06be30 2024-05-09T07:39:40.934 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.924+0000 7f50b8ff9640 1 -- 172.21.15.160:0/3201301958 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f50bc0be100 con 0x7f509c06be30 2024-05-09T07:39:40.934 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 0 to deep-scrub 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f509c06be30 msgr2=0x7f509c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f509c06be30 0x7f509c06e2f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f50a4034000 tx=0x7f50a402e000 comp rx=0 tx=0).stop 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 msgr2=0x7f50bc154440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc154440 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00c440 tx=0x7f50ac001e70 comp rx=0 tx=0).stop 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 shutdown_connections 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f509c06be30 0x7f509c06e2f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f50bc0c8b90 0x7f50bc0c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50bc0c5610 0x7f50bc154440 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 --2- 172.21.15.160:0/3201301958 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f50bc0be320 0x7f50bc1ebc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 >> 172.21.15.160:0/3201301958 conn(0x7f50bc072e90 msgr2=0x7f50bc14d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 shutdown_connections 2024-05-09T07:39:40.937 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:40.926+0000 7f50c27ce640 1 -- 172.21.15.160:0/3201301958 wait complete. 2024-05-09T07:39:40.948 INFO:tasks.ceph:Scrubbing osd.1 2024-05-09T07:39:40.948 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:41.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 Processor -- start 2024-05-09T07:39:41.054 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 -- start start 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c1568e0 0x7f3f3c159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3f3c0c5690 0x7f3f3c0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3f3c156d90 con 0x7f3f3c152a80 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3f3c152e60 con 0x7f3f3c0c5690 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.044+0000 7f3f43d96640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c0c80a0 con 0x7f3f3c1568e0 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3f3c0c5690 0x7f3f3c0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3f3c0c5690 0x7f3f3c0c5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34784/0 (socket says 172.21.15.160:34784) 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 -- 172.21.15.160:0/808505688 learned_addr learned my addr 172.21.15.160:0/808505688 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f41331640 1 --2- 172.21.15.160:0/808505688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c1568e0 0x7f3f3c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 -- 172.21.15.160:0/808505688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c1568e0 msgr2=0x7f3f3c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 --2- 172.21.15.160:0/808505688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c1568e0 0x7f3f3c159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.056 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 -- 172.21.15.160:0/808505688 >> v1:172.21.15.188:6789/0 conn(0x7f3f3c152a80 legacy=0x7f3f3c155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.057 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 -- 172.21.15.160:0/808505688 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c0c7ae0 con 0x7f3f3c0c5690 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.045+0000 7f3f42333640 1 --2- 172.21.15.160:0/808505688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c0c5690 0x7f3f3c0c5ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009e50 tx=0x7f3f3402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f0dbb462104ca4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f40b30640 1 -- 172.21.15.160:0/808505688 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f34036070 con 0x7f3f3c0c5690 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f40b30640 1 -- 172.21.15.160:0/808505688 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f34031600 con 0x7f3f3c0c5690 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f40b30640 1 -- 172.21.15.160:0/808505688 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f340319d0 con 0x7f3f3c0c5690 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 -- 172.21.15.160:0/808505688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c0c5690 msgr2=0x7f3f3c0c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 --2- 172.21.15.160:0/808505688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c0c5690 0x7f3f3c0c5ae0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009e50 tx=0x7f3f3402a5f0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 -- 172.21.15.160:0/808505688 shutdown_connections 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 --2- 172.21.15.160:0/808505688 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c0c5690 0x7f3f3c0c5ae0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 --2- 172.21.15.160:0/808505688 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c1568e0 0x7f3f3c159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.058 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 -- 172.21.15.160:0/808505688 >> 172.21.15.160:0/808505688 conn(0x7f3f3c072ec0 msgr2=0x7f3f3c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 -- 172.21.15.160:0/808505688 shutdown_connections 2024-05-09T07:39:41.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.046+0000 7f3f43d96640 1 -- 172.21.15.160:0/808505688 wait complete. 2024-05-09T07:39:41.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.047+0000 7f3f43d96640 1 Processor -- start 2024-05-09T07:39:41.059 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.047+0000 7f3f43d96640 1 -- start start 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f43d96640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c0c5690 0x7f3f3c1eba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f43d96640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c152a80 0x7f3f3c0c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f43d96640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f3c1568e0 0x7f3f3c0ab990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f43d96640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c158df0 con 0x7f3f3c152a80 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f43d96640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3f3c158b10 con 0x7f3f3c1568e0 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f43d96640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3f3c0ad6a0 con 0x7f3f3c0c5690 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c152a80 0x7f3f3c0c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c152a80 0x7f3f3c0c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34800/0 (socket says 172.21.15.160:34800) 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 -- 172.21.15.160:0/787152328 learned_addr learned my addr 172.21.15.160:0/787152328 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.062 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41b32640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c0c5690 0x7f3f3c1eba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f42333640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f3c1568e0 0x7f3f3c0ab990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 -- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c0c5690 msgr2=0x7f3f3c1eba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c0c5690 0x7f3f3c1eba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 -- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f3c1568e0 msgr2=0x7f3f3c0ab990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f3c1568e0 0x7f3f3c0ab990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 -- 172.21.15.160:0/787152328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f3c1ebfb0 con 0x7f3f3c152a80 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f42333640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3f3c1568e0 0x7f3f3c0ab990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41b32640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3f3c0c5690 0x7f3f3c1eba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.048+0000 7f3f41331640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c152a80 0x7f3f3c0c8af0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00d800 tx=0x7f3f2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.049+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00b6c0 con 0x7f3f3c152a80 2024-05-09T07:39:41.063 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.049+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3f2c019070 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.049+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f3c0ac4e0 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.049+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c00ba80 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.049+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f3c0c0100 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.050+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f3c0bebb0 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.050+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3f2c00bc20 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f327fc640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3f2006be30 0x7f3f2006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f3f2c08b690 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f41b32640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3f2006be30 0x7f3f2006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f327fc640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f3f200757d0 0x7f3f20077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 --> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f2c001d30 con 0x7f3f200757d0 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f2c092050 con 0x7f3f3c152a80 2024-05-09T07:39:41.064 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f42333640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f3f200757d0 0x7f3f20077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.051+0000 7f3f41b32640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3f2006be30 0x7f3f2006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f38001700 tx=0x7f3f38009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.052+0000 7f3f42333640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f3f200757d0 0x7f3f20077c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.065 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.052+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== osd.1 v2:172.21.15.160:6816/4171986763 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f2c001d30 con 0x7f3f200757d0 2024-05-09T07:39:41.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.065+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 --> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f3c155c00 con 0x7f3f200757d0 2024-05-09T07:39:41.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f327fc640 1 -- 172.21.15.160:0/787152328 <== osd.1 v2:172.21.15.160:6816/4171986763 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f3c155c00 con 0x7f3f200757d0 2024-05-09T07:39:41.077 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:41.077 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:41.077 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f3f200757d0 msgr2=0x7f3f20077c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6816/4171986763,v1:172.21.15.160:6817/4171986763] conn(0x7f3f200757d0 0x7f3f20077c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3f2006be30 msgr2=0x7f3f2006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3f2006be30 0x7f3f2006e2f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3f38001700 tx=0x7f3f38009040 comp rx=0 tx=0).stop 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c152a80 msgr2=0x7f3f3c0c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 --2- 172.21.15.160:0/787152328 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3f3c152a80 0x7f3f3c0c8af0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00d800 tx=0x7f3f2c00dcc0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f42333640 1 -- 172.21.15.160:0/787152328 reap_dead start 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 shutdown_connections 2024-05-09T07:39:41.078 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 >> 172.21.15.160:0/787152328 conn(0x7f3f3c072ec0 msgr2=0x7f3f3c153060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 shutdown_connections 2024-05-09T07:39:41.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.066+0000 7f3f43d96640 1 -- 172.21.15.160:0/787152328 wait complete. 2024-05-09T07:39:41.087 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-09T07:39:41.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.181+0000 7fdad9729640 1 Processor -- start 2024-05-09T07:39:41.191 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.181+0000 7fdad9729640 1 -- start start 2024-05-09T07:39:41.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.181+0000 7fdad9729640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdad40be320 0x7fdad40c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.192 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.181+0000 7fdad9729640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad4159840 con 0x7fdad40c8b90 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.181+0000 7fdad9729640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdad4159560 con 0x7fdad40c5610 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.181+0000 7fdad9729640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fdad4159310 con 0x7fdad40be320 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad27fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdad40be320 0x7fdad40c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad27fc640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fdad40be320 0x7fdad40c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34812/0 (socket says 172.21.15.160:34812) 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad37fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fdad40c8b90 0x7fdad4155a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34624/0 (socket says 172.21.15.160:34624) 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad27fc640 1 -- 172.21.15.160:0/3841273579 learned_addr learned my addr 172.21.15.160:0/3841273579 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad27fc640 1 -- 172.21.15.160:0/3841273579 >> v1:172.21.15.188:6789/0 conn(0x7fdad40c5610 legacy=0x7fdad4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad27fc640 1 -- 172.21.15.160:0/3841273579 >> v1:172.21.15.160:6789/0 conn(0x7fdad40c8b90 legacy=0x7fdad4155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad1ffb640 1 -- 172.21.15.160:0/3841273579 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 994629884 0 0) 0x7fdad4159840 con 0x7fdad40c8b90 2024-05-09T07:39:41.193 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.182+0000 7fdad27fc640 1 -- 172.21.15.160:0/3841273579 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad40c77d0 con 0x7fdad40be320 2024-05-09T07:39:41.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad27fc640 1 --2- 172.21.15.160:0/3841273579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40be320 0x7fdad40c59f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009870 tx=0x7fdac802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16311ab70addb650 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.194 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad1ffb640 1 -- 172.21.15.160:0/3841273579 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8037070 con 0x7fdad40be320 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad1ffb640 1 -- 172.21.15.160:0/3841273579 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac8032440 con 0x7fdad40be320 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad1ffb640 1 -- 172.21.15.160:0/3841273579 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8032810 con 0x7fdad40be320 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 -- 172.21.15.160:0/3841273579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40be320 msgr2=0x7fdad40c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 --2- 172.21.15.160:0/3841273579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40be320 0x7fdad40c59f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009870 tx=0x7fdac802a180 comp rx=0 tx=0).stop 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 -- 172.21.15.160:0/3841273579 shutdown_connections 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 --2- 172.21.15.160:0/3841273579 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40be320 0x7fdad40c59f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 -- 172.21.15.160:0/3841273579 >> 172.21.15.160:0/3841273579 conn(0x7fdad4072e90 msgr2=0x7fdad40ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 -- 172.21.15.160:0/3841273579 shutdown_connections 2024-05-09T07:39:41.195 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.183+0000 7fdad9729640 1 -- 172.21.15.160:0/3841273579 wait complete. 2024-05-09T07:39:41.197 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.184+0000 7fdad9729640 1 Processor -- start 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.184+0000 7fdad9729640 1 -- start start 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad9729640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad40be320 0x7fdad41e1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad9729640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 0x7fdad41e0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad9729640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad40c8b90 0x7fdad41df800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad9729640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40ad6a0 con 0x7fdad40c5610 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad9729640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fdad40c7d90 con 0x7fdad40c8b90 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad9729640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fdad4152bb0 con 0x7fdad40be320 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 0x7fdad41e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad27fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 0x7fdad41e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34818/0 (socket says 172.21.15.160:34818) 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad27fc640 1 -- 172.21.15.160:0/547008854 learned_addr learned my addr 172.21.15.160:0/547008854 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad37fe640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad40c8b90 0x7fdad41df800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.185+0000 7fdad2ffd640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad40be320 0x7fdad41e1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.198 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad27fc640 1 -- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad40be320 msgr2=0x7fdad41e1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad27fc640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad40be320 0x7fdad41e1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad27fc640 1 -- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad40c8b90 msgr2=0x7fdad41df800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad27fc640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad40c8b90 0x7fdad41df800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad27fc640 1 -- 172.21.15.160:0/547008854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad41dfd40 con 0x7fdad40c5610 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad27fc640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 0x7fdad41e0960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009870 tx=0x7fdac80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac8037040 con 0x7fdad40c5610 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdac8004250 con 0x7fdad40c5610 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad41eae00 con 0x7fdad40c5610 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac803c730 con 0x7fdad40c5610 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.186+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad41eb680 con 0x7fdad40c5610 2024-05-09T07:39:41.199 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.187+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad4069150 con 0x7fdad40c5610 2024-05-09T07:39:41.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.187+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fdac803a070 con 0x7fdad40c5610 2024-05-09T07:39:41.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.188+0000 7fdabb7fe640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdac006be30 0x7fdac006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.188+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fdac80abc20 con 0x7fdad40c5610 2024-05-09T07:39:41.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.188+0000 7fdad2ffd640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdac006be30 0x7fdac006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.200 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.188+0000 7fdad2ffd640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdac006be30 0x7fdac006e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007fd0 tx=0x7fdac4009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.202 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.192+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fdad4069150 con 0x7fdad40c5610 2024-05-09T07:39:41.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.375+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fdad40756c0 con 0x7fdac006be30 2024-05-09T07:39:41.386 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.375+0000 7fdabb7fe640 1 -- 172.21.15.160:0/547008854 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fdad40756c0 con 0x7fdac006be30 2024-05-09T07:39:41.386 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 1 to deep-scrub 2024-05-09T07:39:41.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdac006be30 msgr2=0x7fdac006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.388 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdac006be30 0x7fdac006e2f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdac4007fd0 tx=0x7fdac4009040 comp rx=0 tx=0).stop 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 msgr2=0x7fdad41e0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 0x7fdad41e0960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdac8009870 tx=0x7fdac80046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 shutdown_connections 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fdac006be30 0x7fdac006e2f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fdad40c8b90 0x7fdad41df800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdad40c5610 0x7fdad41e0960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 --2- 172.21.15.160:0/547008854 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fdad40be320 0x7fdad41e1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.377+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 >> 172.21.15.160:0/547008854 conn(0x7fdad4072e90 msgr2=0x7fdad414e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.378+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 shutdown_connections 2024-05-09T07:39:41.389 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.378+0000 7fdad9729640 1 -- 172.21.15.160:0/547008854 wait complete. 2024-05-09T07:39:41.399 INFO:tasks.ceph:Scrubbing osd.2 2024-05-09T07:39:41.399 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:41.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.497+0000 7fee12fee640 1 Processor -- start 2024-05-09T07:39:41.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.497+0000 7fee12fee640 1 -- start start 2024-05-09T07:39:41.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee12fee640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee0c152a80 0x7fee0c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee12fee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fee0c155e70 0x7fee0c159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee12fee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c0c5690 0x7fee0c0c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee12fee640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c152e60 con 0x7fee0c152a80 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee12fee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fee0c0c80a0 con 0x7fee0c155e70 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee12fee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c0c7ae0 con 0x7fee0c0c5690 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee0c152a80 0x7fee0c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fee0c152a80 0x7fee0c1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34834/0 (socket says 172.21.15.160:34834) 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 -- 172.21.15.160:0/1941514211 learned_addr learned my addr 172.21.15.160:0/1941514211 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee0bfff640 1 --2- 172.21.15.160:0/1941514211 >> v2:172.21.15.188:3300/0 conn(0x7fee0c155e70 0x7fee0c159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee1158b640 1 --2- 172.21.15.160:0/1941514211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c0c5690 0x7fee0c0c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 -- 172.21.15.160:0/1941514211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c0c5690 msgr2=0x7fee0c0c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:41.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 --2- 172.21.15.160:0/1941514211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c0c5690 0x7fee0c0c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 -- 172.21.15.160:0/1941514211 >> v2:172.21.15.188:3300/0 conn(0x7fee0c155e70 msgr2=0x7fee0c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 --2- 172.21.15.160:0/1941514211 >> v2:172.21.15.188:3300/0 conn(0x7fee0c155e70 0x7fee0c159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.498+0000 7fee10d8a640 1 -- 172.21.15.160:0/1941514211 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0c7dc0 con 0x7fee0c152a80 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.499+0000 7fee10d8a640 1 --2- 172.21.15.160:0/1941514211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c152a80 0x7fee0c1568e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0c8220 tx=0x7fee0002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dde24e5d7187f39d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.499+0000 7fee0b7fe640 1 -- 172.21.15.160:0/1941514211 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00037070 con 0x7fee0c152a80 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.499+0000 7fee0b7fe640 1 -- 172.21.15.160:0/1941514211 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee0002adf0 con 0x7fee0c152a80 2024-05-09T07:39:41.511 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.499+0000 7fee0b7fe640 1 -- 172.21.15.160:0/1941514211 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00032660 con 0x7fee0c152a80 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 -- 172.21.15.160:0/1941514211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c152a80 msgr2=0x7fee0c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 --2- 172.21.15.160:0/1941514211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c152a80 0x7fee0c1568e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fee0c0c8220 tx=0x7fee0002a0a0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 -- 172.21.15.160:0/1941514211 shutdown_connections 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 --2- 172.21.15.160:0/1941514211 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c0c5690 0x7fee0c0c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 --2- 172.21.15.160:0/1941514211 >> v2:172.21.15.188:3300/0 conn(0x7fee0c155e70 0x7fee0c159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 --2- 172.21.15.160:0/1941514211 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c152a80 0x7fee0c1568e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 -- 172.21.15.160:0/1941514211 >> 172.21.15.160:0/1941514211 conn(0x7fee0c072ec0 msgr2=0x7fee0c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 -- 172.21.15.160:0/1941514211 shutdown_connections 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.500+0000 7fee12fee640 1 -- 172.21.15.160:0/1941514211 wait complete. 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.501+0000 7fee12fee640 1 Processor -- start 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.501+0000 7fee12fee640 1 -- start start 2024-05-09T07:39:41.514 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee12fee640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c0c5690 0x7fee0c0ab720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee12fee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c152a80 0x7fee0c1ed790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee12fee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0c155e70 0x7fee0c1ec490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee12fee640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c156320 con 0x7fee0c0c5690 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee12fee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fee0c158df0 con 0x7fee0c155e70 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee12fee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fee0c0ad6a0 con 0x7fee0c152a80 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c0c5690 0x7fee0c0ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c0c5690 0x7fee0c0ab720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34848/0 (socket says 172.21.15.160:34848) 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 -- 172.21.15.160:0/1464744393 learned_addr learned my addr 172.21.15.160:0/1464744393 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee0bfff640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c152a80 0x7fee0c1ed790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee1158b640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0c155e70 0x7fee0c1ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 -- 172.21.15.160:0/1464744393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c152a80 msgr2=0x7fee0c1ed790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fee0c152a80 0x7fee0c1ed790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 -- 172.21.15.160:0/1464744393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0c155e70 msgr2=0x7fee0c1ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fee0c155e70 0x7fee0c1ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.515 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 -- 172.21.15.160:0/1464744393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0c0ac620 con 0x7fee0c0c5690 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee10d8a640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c0c5690 0x7fee0c0ab720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fee00000c00 tx=0x7fee000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00037040 con 0x7fee0c0c5690 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.502+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee00042070 con 0x7fee0c0c5690 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.503+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0c1ee5e0 con 0x7fee0c0c5690 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.503+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0003b610 con 0x7fee0c0c5690 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.503+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee0c1eeba0 con 0x7fee0c0c5690 2024-05-09T07:39:41.516 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.503+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fee0c075650 con 0x7fee0c0c5690 2024-05-09T07:39:41.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.504+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fee0003a070 con 0x7fee0c0c5690 2024-05-09T07:39:41.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.504+0000 7fee097fa640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fedf006be60 0x7fedf006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.517 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fee000aba80 con 0x7fee0c0c5690 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee0bfff640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fedf006be60 0x7fedf006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee097fa640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7fedf0075770 0x7fedf0077c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 --> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fee0003ee50 con 0x7fedf0075770 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fee000abf10 con 0x7fee0c0c5690 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee1158b640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7fedf0075770 0x7fedf0077c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee0bfff640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fedf006be60 0x7fedf006e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee0c14e330 tx=0x7fedf8008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.505+0000 7fee1158b640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7fedf0075770 0x7fedf0077c10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.518 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.506+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== osd.2 v2:172.21.15.160:6808/2850827028 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fee0003ee50 con 0x7fedf0075770 2024-05-09T07:39:41.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.519+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 --> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fee0c0ad3a0 con 0x7fedf0075770 2024-05-09T07:39:41.530 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee097fa640 1 -- 172.21.15.160:0/1464744393 <== osd.2 v2:172.21.15.160:6808/2850827028 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fee0c0ad3a0 con 0x7fedf0075770 2024-05-09T07:39:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:41.531 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:41.531 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7fedf0075770 msgr2=0x7fedf0077c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6808/2850827028,v1:172.21.15.160:6809/2850827028] conn(0x7fedf0075770 0x7fedf0077c10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fedf006be60 msgr2=0x7fedf006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fedf006be60 0x7fedf006e320 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fee0c14e330 tx=0x7fedf8008040 comp rx=0 tx=0).stop 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c0c5690 msgr2=0x7fee0c0ab720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 --2- 172.21.15.160:0/1464744393 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fee0c0c5690 0x7fee0c0ab720 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fee00000c00 tx=0x7fee000046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee1158b640 1 -- 172.21.15.160:0/1464744393 reap_dead start 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 shutdown_connections 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 >> 172.21.15.160:0/1464744393 conn(0x7fee0c072ec0 msgr2=0x7fee0c0ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 shutdown_connections 2024-05-09T07:39:41.532 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.520+0000 7fee12fee640 1 -- 172.21.15.160:0/1464744393 wait complete. 2024-05-09T07:39:41.541 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-09T07:39:41.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.634+0000 7f548905d640 1 Processor -- start 2024-05-09T07:39:41.645 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.634+0000 7f548905d640 1 -- start start 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548905d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f54840c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548905d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f54840c8b90 0x7f5484155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548905d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5484159840 con 0x7f54840c5610 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548905d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f5484159310 con 0x7f54840c8b90 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548905d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5484159560 con 0x7f54840be320 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548359e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f54840c8b90 0x7f5484155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f5482d9d640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f54840c5610 0x7f5484155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34628/0 (socket says 172.21.15.160:34628) 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f5482d9d640 1 -- 172.21.15.160:0/889690047 learned_addr learned my addr 172.21.15.160:0/889690047 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f547bfff640 1 --2- 172.21.15.160:0/889690047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f54840c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548359e640 1 -- 172.21.15.160:0/889690047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 msgr2=0x7f54840c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548359e640 1 --2- 172.21.15.160:0/889690047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f54840c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.647 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.635+0000 7f548359e640 1 -- 172.21.15.160:0/889690047 >> v1:172.21.15.160:6789/0 conn(0x7f54840c5610 legacy=0x7f5484155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.636+0000 7f548359e640 1 -- 172.21.15.160:0/889690047 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54840c77d0 con 0x7f54840c8b90 2024-05-09T07:39:41.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.636+0000 7f548259c640 1 -- 172.21.15.160:0/889690047 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 510891731 0 0) 0x7f5484159840 con 0x7f54840c5610 2024-05-09T07:39:41.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.636+0000 7f548359e640 1 --2- 172.21.15.160:0/889690047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c8b90 0x7f5484155a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f547c009d90 tx=0x7f547c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11ac302638a6fac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.636+0000 7f548259c640 1 -- 172.21.15.160:0/889690047 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c007740 con 0x7f54840c8b90 2024-05-09T07:39:41.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.636+0000 7f548259c640 1 -- 172.21.15.160:0/889690047 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f547c0078e0 con 0x7f54840c8b90 2024-05-09T07:39:41.648 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.636+0000 7f548259c640 1 -- 172.21.15.160:0/889690047 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f547c007cb0 con 0x7f54840c8b90 2024-05-09T07:39:41.650 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 -- 172.21.15.160:0/889690047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c8b90 msgr2=0x7f5484155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 --2- 172.21.15.160:0/889690047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c8b90 0x7f5484155a20 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f547c009d90 tx=0x7f547c02a530 comp rx=0 tx=0).stop 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 -- 172.21.15.160:0/889690047 shutdown_connections 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 --2- 172.21.15.160:0/889690047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c8b90 0x7f5484155a20 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 --2- 172.21.15.160:0/889690047 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f54840c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 -- 172.21.15.160:0/889690047 >> 172.21.15.160:0/889690047 conn(0x7f5484072e90 msgr2=0x7f54840ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 -- 172.21.15.160:0/889690047 shutdown_connections 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.637+0000 7f548905d640 1 -- 172.21.15.160:0/889690047 wait complete. 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.638+0000 7f548905d640 1 Processor -- start 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.638+0000 7f548905d640 1 -- start start 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548905d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f5484154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548905d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 0x7f54840c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548905d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54840c8b90 0x7f54840c0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548905d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f54840ad6a0 con 0x7f54840c5610 2024-05-09T07:39:41.651 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548905d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f54840c7d90 con 0x7f54840c8b90 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548905d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5484155f60 con 0x7f54840be320 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 0x7f54840c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 0x7f54840c0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34866/0 (socket says 172.21.15.160:34866) 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 -- 172.21.15.160:0/83053387 learned_addr learned my addr 172.21.15.160:0/83053387 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f5482d9d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f5484154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f548359e640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54840c8b90 0x7f54840c0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 -- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 msgr2=0x7f5484154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f5484154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 -- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54840c8b90 msgr2=0x7f54840c0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54840c8b90 0x7f54840c0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.639+0000 7f547bfff640 1 -- 172.21.15.160:0/83053387 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54841eb970 con 0x7f54840c5610 2024-05-09T07:39:41.652 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.640+0000 7f547bfff640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 0x7f54840c0230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e980 tx=0x7f546c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.641+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c017070 con 0x7f54840c5610 2024-05-09T07:39:41.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.642+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f546c00ca00 con 0x7f54840c5610 2024-05-09T07:39:41.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.642+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c00cdd0 con 0x7f54840c5610 2024-05-09T07:39:41.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.642+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54841ead60 con 0x7f54840c5610 2024-05-09T07:39:41.653 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.642+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841eb320 con 0x7f54840c5610 2024-05-09T07:39:41.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.643+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54840756c0 con 0x7f54840c5610 2024-05-09T07:39:41.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.643+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f546c01a070 con 0x7f54840c5610 2024-05-09T07:39:41.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.643+0000 7f547b7fe640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f546406be30 0x7f546406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.644+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f546c08b150 con 0x7f54840c5610 2024-05-09T07:39:41.654 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.644+0000 7f5482d9d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f546406be30 0x7f546406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.655 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.644+0000 7f5482d9d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f546406be30 0x7f546406e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5474000f60 tx=0x7f5474007d50 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.658 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.648+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f546c059150 con 0x7f54840c5610 2024-05-09T07:39:41.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.832+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f54840ad340 con 0x7f546406be30 2024-05-09T07:39:41.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.833+0000 7f547b7fe640 1 -- 172.21.15.160:0/83053387 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f54840ad340 con 0x7f546406be30 2024-05-09T07:39:41.843 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 2 to deep-scrub 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f546406be30 msgr2=0x7f546406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f546406be30 0x7f546406e2f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5474000f60 tx=0x7f5474007d50 comp rx=0 tx=0).stop 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 msgr2=0x7f54840c0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 0x7f54840c0230 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f546c00e980 tx=0x7f546c00ee40 comp rx=0 tx=0).stop 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 shutdown_connections 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f546406be30 0x7f546406e2f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.846 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f54840c8b90 0x7f54840c0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f54840c5610 0x7f54840c0230 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 --2- 172.21.15.160:0/83053387 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f54840be320 0x7f5484154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 >> 172.21.15.160:0/83053387 conn(0x7f5484072e90 msgr2=0x7f548414f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 shutdown_connections 2024-05-09T07:39:41.847 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.835+0000 7f548905d640 1 -- 172.21.15.160:0/83053387 wait complete. 2024-05-09T07:39:41.857 INFO:tasks.ceph:Scrubbing osd.3 2024-05-09T07:39:41.857 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:41.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.952+0000 7f6010d9d640 1 Processor -- start 2024-05-09T07:39:41.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.952+0000 7f6010d9d640 1 -- start start 2024-05-09T07:39:41.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f6010d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c152a80 0x7f600c1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f6010d9d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f600c0c7ae0 con 0x7f600c155e70 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f6010d9d640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f600c0c7dc0 con 0x7f600c0c5690 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f6010d9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f600c152e60 con 0x7f600c152a80 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f6009d9b640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f600c155e70 0x7f600c159310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34642/0 (socket says 172.21.15.160:34642) 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f6009d9b640 1 -- 172.21.15.160:0/3621187335 learned_addr learned my addr 172.21.15.160:0/3621187335 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.953+0000 7f600a59c640 1 --2- 172.21.15.160:0/3621187335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c152a80 0x7f600c1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2432135069 0 0) 0x7f600c0c7ae0 con 0x7f600c155e70 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5ff8003520 con 0x7f600c155e70 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3822905783 0 0) 0x7f600c0c7dc0 con 0x7f600c0c5690 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f600c0c7ae0 con 0x7f600c0c5690 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600a59c640 1 -- 172.21.15.160:0/3621187335 >> v1:172.21.15.188:6789/0 conn(0x7f600c0c5690 legacy=0x7f600c15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600a59c640 1 -- 172.21.15.160:0/3621187335 >> v1:172.21.15.160:6789/0 conn(0x7f600c155e70 legacy=0x7f600c159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.965 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.954+0000 7f600a59c640 1 -- 172.21.15.160:0/3621187335 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c158b10 con 0x7f600c152a80 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f600a59c640 1 --2- 172.21.15.160:0/3621187335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c152a80 0x7f600c1568e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40097d0 tx=0x7f5ff402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6847db10e9e0b920 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 <== mon.2 v2:172.21.15.188:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4037070 con 0x7f600c152a80 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 <== mon.2 v2:172.21.15.188:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff402ae50 con 0x7f600c152a80 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f600959a640 1 -- 172.21.15.160:0/3621187335 <== mon.2 v2:172.21.15.188:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4032640 con 0x7f600c152a80 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 -- 172.21.15.160:0/3621187335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c152a80 msgr2=0x7f600c1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 --2- 172.21.15.160:0/3621187335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c152a80 0x7f600c1568e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5ff40097d0 tx=0x7f5ff402a100 comp rx=0 tx=0).stop 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 -- 172.21.15.160:0/3621187335 shutdown_connections 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 --2- 172.21.15.160:0/3621187335 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c152a80 0x7f600c1568e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 -- 172.21.15.160:0/3621187335 >> 172.21.15.160:0/3621187335 conn(0x7f600c072ec0 msgr2=0x7f600c0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 -- 172.21.15.160:0/3621187335 shutdown_connections 2024-05-09T07:39:41.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.955+0000 7f6010d9d640 1 -- 172.21.15.160:0/3621187335 wait complete. 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 Processor -- start 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 -- start start 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0c5690 0x7f600c1ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c152a80 0x7f600c0c2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c155e70 0x7f600c0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0ad6a0 con 0x7f600c0c5690 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f600c0c80a0 con 0x7f600c152a80 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f6010d9d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f600c1562f0 con 0x7f600c155e70 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f600a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0c5690 0x7f600c1ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f600a59c640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0c5690 0x7f600c1ee640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34868/0 (socket says 172.21.15.160:34868) 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.957+0000 7f600a59c640 1 -- 172.21.15.160:0/3496912554 learned_addr learned my addr 172.21.15.160:0/3496912554 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f6009d9b640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c152a80 0x7f600c0c2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600ad9d640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c155e70 0x7f600c0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600a59c640 1 -- 172.21.15.160:0/3496912554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c155e70 msgr2=0x7f600c0c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600a59c640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f600c155e70 0x7f600c0c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600a59c640 1 -- 172.21.15.160:0/3496912554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c152a80 msgr2=0x7f600c0c2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600a59c640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f600c152a80 0x7f600c0c2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600a59c640 1 -- 172.21.15.160:0/3496912554 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f600c0c9400 con 0x7f600c0c5690 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f600a59c640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0c5690 0x7f600c1ee640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4035040 tx=0x7f5ff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.971 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4037050 con 0x7f600c0c5690 2024-05-09T07:39:41.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ff4038040 con 0x7f600c0c5690 2024-05-09T07:39:41.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f600c0be2c0 con 0x7f600c0c5690 2024-05-09T07:39:41.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40044c0 con 0x7f600c0c5690 2024-05-09T07:39:41.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.958+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f600c0c2890 con 0x7f600c0c5690 2024-05-09T07:39:41.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.959+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f600c0c7690 con 0x7f600c0c5690 2024-05-09T07:39:41.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5ff4045020 con 0x7f600c0c5690 2024-05-09T07:39:41.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6002ffd640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5fec06be30 0x7f5fec06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5ff40aafa0 con 0x7f600c0c5690 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6009d9b640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5fec06be30 0x7f5fec06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6002ffd640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f5fec0757d0 0x7f5fec077c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 --> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ff403fe50 con 0x7f5fec0757d0 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5ff40b2050 con 0x7f600c0c5690 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f600ad9d640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f5fec0757d0 0x7f5fec077c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.960+0000 7f6009d9b640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5fec06be30 0x7f5fec06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f600c0c1a40 tx=0x7f5ffc007d70 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.961+0000 7f600ad9d640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f5fec0757d0 0x7f5fec077c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:41.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.961+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== osd.3 v2:172.21.15.160:6824/3093825945 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ff403fe50 con 0x7f5fec0757d0 2024-05-09T07:39:41.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 --> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f600c075650 con 0x7f5fec0757d0 2024-05-09T07:39:41.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6002ffd640 1 -- 172.21.15.160:0/3496912554 <== osd.3 v2:172.21.15.160:6824/3093825945 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f600c075650 con 0x7f5fec0757d0 2024-05-09T07:39:41.986 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:41.986 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:41.986 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f5fec0757d0 msgr2=0x7f5fec077c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6824/3093825945,v1:172.21.15.160:6825/3093825945] conn(0x7f5fec0757d0 0x7f5fec077c70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5fec06be30 msgr2=0x7f5fec06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5fec06be30 0x7f5fec06e2f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f600c0c1a40 tx=0x7f5ffc007d70 comp rx=0 tx=0).stop 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0c5690 msgr2=0x7f600c1ee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 --2- 172.21.15.160:0/3496912554 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f600c0c5690 0x7f600c1ee640 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4035040 tx=0x7f5ff40046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f600ad9d640 1 -- 172.21.15.160:0/3496912554 reap_dead start 2024-05-09T07:39:41.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 shutdown_connections 2024-05-09T07:39:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 >> 172.21.15.160:0/3496912554 conn(0x7f600c072ec0 msgr2=0x7f600c0aa6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 shutdown_connections 2024-05-09T07:39:41.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:41.975+0000 7f6010d9d640 1 -- 172.21.15.160:0/3496912554 wait complete. 2024-05-09T07:39:42.000 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-09T07:39:42.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.092+0000 7f5eb33f9640 1 Processor -- start 2024-05-09T07:39:42.103 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.092+0000 7f5eb33f9640 1 -- start start 2024-05-09T07:39:42.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb33f9640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eac0be320 0x7f5eac0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb33f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c8b90 0x7f5eac155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.104 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb33f9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5eac159840 con 0x7f5eac0c5610 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb33f9640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f5eac159310 con 0x7f5eac0be320 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb33f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac159560 con 0x7f5eac0c8b90 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb0994640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f5eac0be320 0x7f5eac0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb1996640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c8b90 0x7f5eac155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.093+0000 7f5eb1195640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f5eac0c5610 0x7f5eac155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:45032/0 (socket says 172.21.15.160:45032) 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.094+0000 7f5eb1195640 1 -- 172.21.15.160:0/4164875957 learned_addr learned my addr 172.21.15.160:0/4164875957 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.094+0000 7f5eb0994640 1 -- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c8b90 msgr2=0x7f5eac155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.094+0000 7f5eb0994640 1 --2- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c8b90 0x7f5eac155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.094+0000 7f5eb0994640 1 -- 172.21.15.160:0/4164875957 >> v1:172.21.15.188:6789/0 conn(0x7f5eac0c5610 legacy=0x7f5eac155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.094+0000 7f5ea3fff640 1 -- 172.21.15.160:0/4164875957 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 378631340 0 0) 0x7f5eac159840 con 0x7f5eac0c5610 2024-05-09T07:39:42.105 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.094+0000 7f5eb0994640 1 -- 172.21.15.160:0/4164875957 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac0c77d0 con 0x7f5eac0be320 2024-05-09T07:39:42.106 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb0994640 1 --2- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0be320 0x7f5eac0c59f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80097a0 tx=0x7f5ea802a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d9fb013e743d3e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5ea3fff640 1 -- 172.21.15.160:0/4164875957 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8037070 con 0x7f5eac0be320 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5ea3fff640 1 -- 172.21.15.160:0/4164875957 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ea802adb0 con 0x7f5eac0be320 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5ea3fff640 1 -- 172.21.15.160:0/4164875957 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8032660 con 0x7f5eac0be320 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 -- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0be320 msgr2=0x7f5eac0c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0be320 0x7f5eac0c59f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5ea80097a0 tx=0x7f5ea802a060 comp rx=0 tx=0).stop 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 -- 172.21.15.160:0/4164875957 shutdown_connections 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0be320 0x7f5eac0c59f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/4164875957 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c8b90 0x7f5eac155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 -- 172.21.15.160:0/4164875957 >> 172.21.15.160:0/4164875957 conn(0x7f5eac072e90 msgr2=0x7f5eac0ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 -- 172.21.15.160:0/4164875957 shutdown_connections 2024-05-09T07:39:42.107 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.095+0000 7f5eb33f9640 1 -- 172.21.15.160:0/4164875957 wait complete. 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.096+0000 7f5eb33f9640 1 Processor -- start 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 -- start start 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 0x7f5eac157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c5610 0x7f5eac148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0c8b90 0x7f5eac1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0ad6a0 con 0x7f5eac0be320 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f5eac0c7d90 con 0x7f5eac0c8b90 2024-05-09T07:39:42.110 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb33f9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f5eac155f60 con 0x7f5eac0c5610 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb1195640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 0x7f5eac157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb1195640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 0x7f5eac157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34882/0 (socket says 172.21.15.160:34882) 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb1195640 1 -- 172.21.15.160:0/171098210 learned_addr learned my addr 172.21.15.160:0/171098210 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb0994640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c5610 0x7f5eac148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.097+0000 7f5eb1996640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0c8b90 0x7f5eac1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb1195640 1 -- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c5610 msgr2=0x7f5eac148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb1195640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c5610 0x7f5eac148f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb1195640 1 -- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0c8b90 msgr2=0x7f5eac1494c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb1195640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0c8b90 0x7f5eac1494c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb1195640 1 -- 172.21.15.160:0/171098210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5eac156880 con 0x7f5eac0be320 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb1195640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 0x7f5eac157610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c500 tx=0x7f5e9c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.111 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c018070 con 0x7f5eac0be320 2024-05-09T07:39:42.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e9c007580 con 0x7f5eac0be320 2024-05-09T07:39:42.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c004a20 con 0x7f5eac0be320 2024-05-09T07:39:42.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5eac1565e0 con 0x7f5eac0be320 2024-05-09T07:39:42.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.098+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5eac14c130 con 0x7f5eac0be320 2024-05-09T07:39:42.112 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.099+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5eac0756c0 con 0x7f5eac0be320 2024-05-09T07:39:42.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.100+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f5e9c01b070 con 0x7f5eac0be320 2024-05-09T07:39:42.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.100+0000 7f5ea1ffb640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e9006be30 0x7f5e9006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.100+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f5e9c08a480 con 0x7f5eac0be320 2024-05-09T07:39:42.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.100+0000 7f5eb0994640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e9006be30 0x7f5e9006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.113 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.101+0000 7f5eb0994640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e9006be30 0x7f5e9006e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8034000 tx=0x7f5ea802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.115 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.104+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5e9c058480 con 0x7f5eac0be320 2024-05-09T07:39:42.296 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.286+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f5eac0ca1a0 con 0x7f5e9006be30 2024-05-09T07:39:42.297 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.286+0000 7f5ea1ffb640 1 -- 172.21.15.160:0/171098210 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5eac0ca1a0 con 0x7f5e9006be30 2024-05-09T07:39:42.297 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 3 to deep-scrub 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e9006be30 msgr2=0x7f5e9006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e9006be30 0x7f5e9006e2f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5ea8034000 tx=0x7f5ea802e000 comp rx=0 tx=0).stop 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 msgr2=0x7f5eac157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 0x7f5eac157610 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c500 tx=0x7f5e9c00c9c0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 shutdown_connections 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f5e9006be30 0x7f5e9006e2f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f5eac0c8b90 0x7f5eac1494c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f5eac0c5610 0x7f5eac148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 --2- 172.21.15.160:0/171098210 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5eac0be320 0x7f5eac157610 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 >> 172.21.15.160:0/171098210 conn(0x7f5eac072e90 msgr2=0x7f5eac154b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 shutdown_connections 2024-05-09T07:39:42.300 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.289+0000 7f5eb33f9640 1 -- 172.21.15.160:0/171098210 wait complete. 2024-05-09T07:39:42.311 INFO:tasks.ceph:Scrubbing osd.4 2024-05-09T07:39:42.311 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:42.419 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.409+0000 7fbe009d5640 1 Processor -- start 2024-05-09T07:39:42.420 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.409+0000 7fbe009d5640 1 -- start start 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbe009d5640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbdfc1595d0 0x7fbdfc0c2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbe009d5640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdfc0c3fa0 con 0x7fbdfc156200 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbe009d5640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbdfc0c4280 con 0x7fbdfc0c8aa0 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbe009d5640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fbdfc0c4560 con 0x7fbdfc1595d0 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdf9d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbdfc1595d0 0x7fbdfc0c2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdfa59c640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fbdfc156200 0x7fbdfc0c1e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:34648/0 (socket says 172.21.15.160:34648) 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdf9d9b640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fbdfc1595d0 0x7fbdfc0c2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34898/0 (socket says 172.21.15.160:34898) 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdfa59c640 1 -- 172.21.15.160:0/489258785 learned_addr learned my addr 172.21.15.160:0/489258785 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdf9d9b640 1 -- 172.21.15.160:0/489258785 >> v1:172.21.15.188:6789/0 conn(0x7fbdfc0c8aa0 legacy=0x7fbdfc15fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdf9d9b640 1 -- 172.21.15.160:0/489258785 >> v1:172.21.15.160:6789/0 conn(0x7fbdfc156200 legacy=0x7fbdfc0c1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.421 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.410+0000 7fbdf9d9b640 1 -- 172.21.15.160:0/489258785 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc0c8580 con 0x7fbdfc1595d0 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.411+0000 7fbdf9d9b640 1 --2- 172.21.15.160:0/489258785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc0c2930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbde40097a0 tx=0x7fbde402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5e1e9dc8e116f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.411+0000 7fbdf959a640 1 -- 172.21.15.160:0/489258785 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4037070 con 0x7fbdfc1595d0 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.411+0000 7fbdf959a640 1 -- 172.21.15.160:0/489258785 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbde4032440 con 0x7fbdfc1595d0 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.411+0000 7fbdf959a640 1 -- 172.21.15.160:0/489258785 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde4032810 con 0x7fbdfc1595d0 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 -- 172.21.15.160:0/489258785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 msgr2=0x7fbdfc0c2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 --2- 172.21.15.160:0/489258785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc0c2930 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbde40097a0 tx=0x7fbde402a230 comp rx=0 tx=0).stop 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 -- 172.21.15.160:0/489258785 shutdown_connections 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 --2- 172.21.15.160:0/489258785 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc0c2930 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 -- 172.21.15.160:0/489258785 >> 172.21.15.160:0/489258785 conn(0x7fbdfc0aa810 msgr2=0x7fbdfc0aabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 -- 172.21.15.160:0/489258785 shutdown_connections 2024-05-09T07:39:42.423 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.412+0000 7fbe009d5640 1 -- 172.21.15.160:0/489258785 wait complete. 2024-05-09T07:39:42.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.413+0000 7fbe009d5640 1 Processor -- start 2024-05-09T07:39:42.424 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.413+0000 7fbe009d5640 1 -- start start 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbe009d5640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc0c8aa0 0x7fbdfc1df900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbe009d5640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc156200 0x7fbdfc0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbe009d5640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc1e0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbe009d5640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0c37c0 con 0x7fbdfc1595d0 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbe009d5640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fbdfc0c3640 con 0x7fbdfc156200 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbe009d5640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fbdfc1565e0 con 0x7fbdfc0c8aa0 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc1e0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc1e0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34900/0 (socket says 172.21.15.160:34900) 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 -- 172.21.15.160:0/2836014726 learned_addr learned my addr 172.21.15.160:0/2836014726 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdf9d9b640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc156200 0x7fbdfc0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfa59c640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc0c8aa0 0x7fbdfc1df900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 -- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc0c8aa0 msgr2=0x7fbdfc1df900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fbdfc0c8aa0 0x7fbdfc1df900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.427 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 -- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc156200 msgr2=0x7fbdfc0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fbdfc156200 0x7fbdfc0add40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 -- 172.21.15.160:0/2836014726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdfc1eb2c0 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdfad9d640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc1e0b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c7a0 tx=0x7fbdf400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.414+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf4003b70 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.415+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdf4003d30 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.415+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdfc1dee20 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.415+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400aa90 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.415+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdfc0ae280 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.415+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbdfc075650 con 0x7fbdfc1595d0 2024-05-09T07:39:42.428 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.416+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fbdf400ac30 con 0x7fbdfc1595d0 2024-05-09T07:39:42.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdf2ffd640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbddc06be60 0x7fbddc06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fbdf408b0b0 con 0x7fbdfc1595d0 2024-05-09T07:39:42.429 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdfa59c640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbddc06be60 0x7fbddc06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdf2ffd640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7fbddc0757f0 0x7fbddc077c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 --> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbdf4010e50 con 0x7fbddc0757f0 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fbdf4092050 con 0x7fbdfc1595d0 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdfa59c640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbddc06be60 0x7fbddc06e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005880 tx=0x7fbdec009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.417+0000 7fbdf9d9b640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7fbddc0757f0 0x7fbddc077c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.418+0000 7fbdf9d9b640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7fbddc0757f0 0x7fbddc077c90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.430 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.418+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== osd.4 v2:172.21.15.188:6800/2865253491 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbdf4010e50 con 0x7fbddc0757f0 2024-05-09T07:39:42.442 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 --> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbdfc06c310 con 0x7fbddc0757f0 2024-05-09T07:39:42.443 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbdf2ffd640 1 -- 172.21.15.160:0/2836014726 <== osd.4 v2:172.21.15.188:6800/2865253491 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbdfc06c310 con 0x7fbddc0757f0 2024-05-09T07:39:42.443 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:42.443 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:42.443 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7fbddc0757f0 msgr2=0x7fbddc077c90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.188:6800/2865253491,v1:172.21.15.188:6801/2865253491] conn(0x7fbddc0757f0 0x7fbddc077c90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbddc06be60 msgr2=0x7fbddc06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fbddc06be60 0x7fbddc06e320 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbdec005880 tx=0x7fbdec009040 comp rx=0 tx=0).stop 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 msgr2=0x7fbdfc1e0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 --2- 172.21.15.160:0/2836014726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdfc1595d0 0x7fbdfc1e0b20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fbdf400c7a0 tx=0x7fbdf400cc60 comp rx=0 tx=0).stop 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbdfad9d640 1 -- 172.21.15.160:0/2836014726 reap_dead start 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 shutdown_connections 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 >> 172.21.15.160:0/2836014726 conn(0x7fbdfc0aa810 msgr2=0x7fbdfc0c7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.444 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 shutdown_connections 2024-05-09T07:39:42.445 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.432+0000 7fbe009d5640 1 -- 172.21.15.160:0/2836014726 wait complete. 2024-05-09T07:39:42.454 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-09T07:39:42.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.546+0000 7f1c89c76640 1 Processor -- start 2024-05-09T07:39:42.557 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.546+0000 7f1c89c76640 1 -- start start 2024-05-09T07:39:42.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c89c76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c84157410 0x7f1c8406f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c89c76640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1c8406fa80 0x7f1c840c71f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.559 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c89c76640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1c8406ff00 0x7f1c840c7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c89c76640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f1c840c3750 con 0x7f1c8406ff00 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c89c76640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1c840c5860 con 0x7f1c8406fa80 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c89c76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1c840c5b40 con 0x7f1c84157410 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1c8406ff00 0x7f1c840c7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1c8406ff00 0x7f1c840c7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48274/0 (socket says 172.21.15.160:48274) 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 -- 172.21.15.160:0/2639868988 learned_addr learned my addr 172.21.15.160:0/2639868988 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c82ffd640 1 --2- 172.21.15.160:0/2639868988 >> v2:172.21.15.188:3300/0 conn(0x7f1c8406fa80 0x7f1c840c71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 -- 172.21.15.160:0/2639868988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c84157410 msgr2=0x7f1c8406f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:42.560 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 --2- 172.21.15.160:0/2639868988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c84157410 0x7f1c8406f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 -- 172.21.15.160:0/2639868988 >> v2:172.21.15.188:3300/0 conn(0x7f1c8406fa80 msgr2=0x7f1c840c71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 --2- 172.21.15.160:0/2639868988 >> v2:172.21.15.188:3300/0 conn(0x7f1c8406fa80 0x7f1c840c71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.547+0000 7f1c83fff640 1 -- 172.21.15.160:0/2639868988 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c840c5e20 con 0x7f1c8406ff00 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.548+0000 7f1c83fff640 1 --2- 172.21.15.160:0/2639868988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c840c7730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1c840c68a0 tx=0x7f1c7c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee51e818cd86d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.548+0000 7f1c827fc640 1 -- 172.21.15.160:0/2639868988 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c036070 con 0x7f1c8406ff00 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.548+0000 7f1c827fc640 1 -- 172.21.15.160:0/2639868988 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c7c031600 con 0x7f1c8406ff00 2024-05-09T07:39:42.561 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.548+0000 7f1c827fc640 1 -- 172.21.15.160:0/2639868988 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c7c0319d0 con 0x7f1c8406ff00 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 -- 172.21.15.160:0/2639868988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 msgr2=0x7f1c840c7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 --2- 172.21.15.160:0/2639868988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c840c7730 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1c840c68a0 tx=0x7f1c7c02a610 comp rx=0 tx=0).stop 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 -- 172.21.15.160:0/2639868988 shutdown_connections 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 --2- 172.21.15.160:0/2639868988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c840c7730 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 --2- 172.21.15.160:0/2639868988 >> v2:172.21.15.188:3300/0 conn(0x7f1c8406fa80 0x7f1c840c71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 --2- 172.21.15.160:0/2639868988 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c84157410 0x7f1c8406f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 -- 172.21.15.160:0/2639868988 >> 172.21.15.160:0/2639868988 conn(0x7f1c84066f90 msgr2=0x7f1c84067370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.562 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 -- 172.21.15.160:0/2639868988 shutdown_connections 2024-05-09T07:39:42.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.549+0000 7f1c89c76640 1 -- 172.21.15.160:0/2639868988 wait complete. 2024-05-09T07:39:42.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.551+0000 7f1c89c76640 1 Processor -- start 2024-05-09T07:39:42.563 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.551+0000 7f1c89c76640 1 -- start start 2024-05-09T07:39:42.565 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c89c76640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c8406fa80 0x7f1c840c28f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c89c76640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c841e9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c89c76640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c84157410 0x7f1c840c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c89c76640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1c84157970 con 0x7f1c8406ff00 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c89c76640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1c841577f0 con 0x7f1c84157410 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c89c76640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1c840c5400 con 0x7f1c8406fa80 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c841e9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c841e9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48282/0 (socket says 172.21.15.160:48282) 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 -- 172.21.15.160:0/4120500928 learned_addr learned my addr 172.21.15.160:0/4120500928 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c83fff640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c84157410 0x7f1c840c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c837fe640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c8406fa80 0x7f1c840c28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 -- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c8406fa80 msgr2=0x7f1c840c28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.566 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c8406fa80 0x7f1c840c28f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 -- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c84157410 msgr2=0x7f1c840c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c84157410 0x7f1c840c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 -- 172.21.15.160:0/4120500928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c841e8ca0 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c83fff640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c84157410 0x7f1c840c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.552+0000 7f1c82ffd640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c841e9760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c7f0 tx=0x7f1c7800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.553+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78015070 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.553+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c78004220 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.553+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c840bde30 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.553+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c78007620 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.553+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c841e8f60 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.554+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c840756c0 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.554+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1c780077c0 con 0x7f1c8406ff00 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.555+0000 7f1c6bfff640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1c7006be30 0x7f1c7006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.567 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.555+0000 7f1c837fe640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1c7006be30 0x7f1c7006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.555+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f1c7808b2d0 con 0x7f1c8406ff00 2024-05-09T07:39:42.568 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.555+0000 7f1c837fe640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1c7006be30 0x7f1c7006e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400a810 tx=0x7f1c74008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.570 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.559+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1c78059920 con 0x7f1c8406ff00 2024-05-09T07:39:42.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.743+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f1c840702e0 con 0x7f1c7006be30 2024-05-09T07:39:42.759 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.744+0000 7f1c6bfff640 1 -- 172.21.15.160:0/4120500928 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1c840702e0 con 0x7f1c7006be30 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 4 to deep-scrub 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1c7006be30 msgr2=0x7f1c7006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1c7006be30 0x7f1c7006e2f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1c7400a810 tx=0x7f1c74008040 comp rx=0 tx=0).stop 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 msgr2=0x7f1c841e9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c841e9760 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1c7800c7f0 tx=0x7f1c7800ccb0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 shutdown_connections 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1c7006be30 0x7f1c7006e2f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1c84157410 0x7f1c840c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c8406ff00 0x7f1c841e9760 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.760 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 --2- 172.21.15.160:0/4120500928 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1c8406fa80 0x7f1c840c28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 >> 172.21.15.160:0/4120500928 conn(0x7f1c84066f90 msgr2=0x7f1c84154c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 shutdown_connections 2024-05-09T07:39:42.761 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.746+0000 7f1c89c76640 1 -- 172.21.15.160:0/4120500928 wait complete. 2024-05-09T07:39:42.770 INFO:tasks.ceph:Scrubbing osd.5 2024-05-09T07:39:42.771 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:42.879 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.869+0000 7fd57d970640 1 Processor -- start 2024-05-09T07:39:42.880 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.869+0000 7fd57d970640 1 -- start start 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd57d970640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd578152a30 0x7fd578070e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd57d970640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd5780c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd57d970640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd578070690 con 0x7fd5780c8aa0 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd57d970640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fd578071360 con 0x7fd578152a30 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd57d970640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780a7d40 con 0x7fd5780c2a00 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd5767fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd5780c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.881 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd576ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd578152a30 0x7fd578070e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd5767fc640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd5780c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3301/0 says I am v2:172.21.15.160:56686/0 (socket says 172.21.15.160:56686) 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd5767fc640 1 -- 172.21.15.160:0/3320379752 learned_addr learned my addr 172.21.15.160:0/3320379752 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd576ffd640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fd578152a30 0x7fd578070e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:35526/0 (socket says 172.21.15.160:35526) 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd576ffd640 1 -- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 msgr2=0x7fd5780c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd576ffd640 1 --2- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd5780c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd576ffd640 1 -- 172.21.15.160:0/3320379752 >> v1:172.21.15.188:6789/0 conn(0x7fd5780c8aa0 legacy=0x7fd5780aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd576ffd640 1 -- 172.21.15.160:0/3320379752 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5780a8020 con 0x7fd578152a30 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd575ffb640 1 -- 172.21.15.160:0/3320379752 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1892403940 0 0) 0x7fd578070690 con 0x7fd5780c8aa0 2024-05-09T07:39:42.882 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.870+0000 7fd5767fc640 1 --2- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd5780c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:39:42.883 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.871+0000 7fd576ffd640 1 --2- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 0x7fd578070e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5680097a0 tx=0x7fd56802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fc3176503602343 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.871+0000 7fd575ffb640 1 -- 172.21.15.160:0/3320379752 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568037070 con 0x7fd578152a30 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.871+0000 7fd575ffb640 1 -- 172.21.15.160:0/3320379752 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd568032440 con 0x7fd578152a30 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.871+0000 7fd575ffb640 1 -- 172.21.15.160:0/3320379752 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd568032810 con 0x7fd578152a30 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 -- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 msgr2=0x7fd578070e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 --2- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 0x7fd578070e20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd5680097a0 tx=0x7fd56802a180 comp rx=0 tx=0).stop 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 -- 172.21.15.160:0/3320379752 shutdown_connections 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 --2- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd5780c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.884 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 --2- 172.21.15.160:0/3320379752 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 0x7fd578070e20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 -- 172.21.15.160:0/3320379752 >> 172.21.15.160:0/3320379752 conn(0x7fd578072a80 msgr2=0x7fd578072e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 -- 172.21.15.160:0/3320379752 shutdown_connections 2024-05-09T07:39:42.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.872+0000 7fd57d970640 1 -- 172.21.15.160:0/3320379752 wait complete. 2024-05-09T07:39:42.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.873+0000 7fd57d970640 1 Processor -- start 2024-05-09T07:39:42.885 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.873+0000 7fd57d970640 1 -- start start 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd57d970640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd57814c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd57d970640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5780c8aa0 0x7fd578159260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd57d970640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 0x7fd578157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd57d970640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fd578152e10 con 0x7fd5780c8aa0 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd57d970640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fd578070900 con 0x7fd578152a30 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd57d970640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fd5780c6240 con 0x7fd5780c2a00 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5780c8aa0 0x7fd578159260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5780c8aa0 0x7fd578159260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48288/0 (socket says 172.21.15.160:48288) 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 -- 172.21.15.160:0/1736485082 learned_addr learned my addr 172.21.15.160:0/1736485082 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5777fe640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 0x7fd578157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd576ffd640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd57814c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.888 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 -- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 msgr2=0x7fd57814c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fd5780c2a00 0x7fd57814c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 -- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 msgr2=0x7fd578157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fd578152a30 0x7fd578157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 -- 172.21.15.160:0/1736485082 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd578153d00 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.874+0000 7fd5767fc640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5780c8aa0 0x7fd578159260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00e9b0 tx=0x7fd56c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.875+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c017070 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.875+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd56c00c9a0 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.875+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5781ebd20 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.875+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00cd50 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.875+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd578156c70 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.875+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd578075650 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.876+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fd56c01a070 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.876+0000 7fd55ffff640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd56406be30 0x7fd56406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fd56c08b110 con 0x7fd5780c8aa0 2024-05-09T07:39:42.889 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd576ffd640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd56406be30 0x7fd56406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd55ffff640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7fd564075740 0x7fd564077be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 --> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5640782b0 con 0x7fd564075740 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd56c092050 con 0x7fd5780c8aa0 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd5777fe640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7fd564075740 0x7fd564077be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd576ffd640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd56406be30 0x7fd56406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd568034000 tx=0x7fd56802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.877+0000 7fd5777fe640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7fd564075740 0x7fd564077be0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:42.890 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.878+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== osd.5 v2:172.21.15.188:6816/173821344 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5640782b0 con 0x7fd564075740 2024-05-09T07:39:42.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.890+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 --> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd5780c1fa0 con 0x7fd564075740 2024-05-09T07:39:42.902 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd55ffff640 1 -- 172.21.15.160:0/1736485082 <== osd.5 v2:172.21.15.188:6816/173821344 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd5780c1fa0 con 0x7fd564075740 2024-05-09T07:39:42.903 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:42.903 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:42.903 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7fd564075740 msgr2=0x7fd564077be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.188:6816/173821344,v1:172.21.15.188:6817/173821344] conn(0x7fd564075740 0x7fd564077be0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd56406be30 msgr2=0x7fd56406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fd56406be30 0x7fd56406e2f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd568034000 tx=0x7fd56802e000 comp rx=0 tx=0).stop 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5780c8aa0 msgr2=0x7fd578159260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 --2- 172.21.15.160:0/1736485082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd5780c8aa0 0x7fd578159260 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00e9b0 tx=0x7fd56c00ee70 comp rx=0 tx=0).stop 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd5777fe640 1 -- 172.21.15.160:0/1736485082 reap_dead start 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 shutdown_connections 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 >> 172.21.15.160:0/1736485082 conn(0x7fd578072a80 msgr2=0x7fd5780c8270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:42.904 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 shutdown_connections 2024-05-09T07:39:42.905 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:42.891+0000 7fd57d970640 1 -- 172.21.15.160:0/1736485082 wait complete. 2024-05-09T07:39:42.914 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-09T07:39:43.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.005+0000 7fc5dd3b7640 1 Processor -- start 2024-05-09T07:39:43.017 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.005+0000 7fc5dd3b7640 1 -- start start 2024-05-09T07:39:43.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5dd3b7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc5d80c5610 0x7fc5d80c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5dd3b7640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fc5d80be320 0x7fc5d8155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.019 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5dd3b7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc5d8159840 con 0x7fc5d80c8b90 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5dd3b7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fc5d8159310 con 0x7fc5d80c5610 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5dd3b7640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fc5d8159560 con 0x7fc5d80be320 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc5d80c5610 0x7fc5d80c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d77fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fc5d80c8b90 0x7fc5d8155850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51924/0 (socket says 172.21.15.160:51924) 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d77fe640 1 -- 172.21.15.160:0/491678291 learned_addr learned my addr 172.21.15.160:0/491678291 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d6ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fc5d80c5610 0x7fc5d80c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48290/0 (socket says 172.21.15.160:48290) 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5cffff640 1 --2- 172.21.15.160:0/491678291 >> v2:172.21.15.188:3300/0 conn(0x7fc5d80be320 0x7fc5d8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/491678291 >> v2:172.21.15.188:3300/0 conn(0x7fc5d80be320 msgr2=0x7fc5d8155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d6ffd640 1 --2- 172.21.15.160:0/491678291 >> v2:172.21.15.188:3300/0 conn(0x7fc5d80be320 0x7fc5d8155310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/491678291 >> v1:172.21.15.160:6789/0 conn(0x7fc5d80c8b90 legacy=0x7fc5d8155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/491678291 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d80c77d0 con 0x7fc5d80c5610 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.006+0000 7fc5d67fc640 1 -- 172.21.15.160:0/491678291 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1517733322 0 0) 0x7fc5d8159840 con 0x7fc5d80c8b90 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.007+0000 7fc5d6ffd640 1 --2- 172.21.15.160:0/491678291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80c5610 0x7fc5d80c59f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5d806d860 tx=0x7fc5c802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c35987f749d6db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.007+0000 7fc5d67fc640 1 -- 172.21.15.160:0/491678291 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8037070 con 0x7fc5d80c5610 2024-05-09T07:39:43.020 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.007+0000 7fc5d67fc640 1 -- 172.21.15.160:0/491678291 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c802add0 con 0x7fc5d80c5610 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.007+0000 7fc5d67fc640 1 -- 172.21.15.160:0/491678291 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8032660 con 0x7fc5d80c5610 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.007+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/491678291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80c5610 msgr2=0x7fc5d80c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.007+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/491678291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80c5610 0x7fc5d80c59f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc5d806d860 tx=0x7fc5c802a080 comp rx=0 tx=0).stop 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.008+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/491678291 shutdown_connections 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.008+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/491678291 >> v2:172.21.15.188:3300/0 conn(0x7fc5d80be320 0x7fc5d8155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.008+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/491678291 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80c5610 0x7fc5d80c59f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.008+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/491678291 >> 172.21.15.160:0/491678291 conn(0x7fc5d8072e90 msgr2=0x7fc5d80ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.008+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/491678291 shutdown_connections 2024-05-09T07:39:43.021 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.008+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/491678291 wait complete. 2024-05-09T07:39:43.023 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 Processor -- start 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 -- start start 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 0x7fc5d80c48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d80c5610 0x7fc5d80ab6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d80c8b90 0x7fc5d80c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d80ad6a0 con 0x7fc5d80be320 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d80c7d90 con 0x7fc5d80c5610 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5dd3b7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fc5d8155f60 con 0x7fc5d80c8b90 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.009+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 0x7fc5d80c48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 0x7fc5d80c48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48294/0 (socket says 172.21.15.160:48294) 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/4195595439 learned_addr learned my addr 172.21.15.160:0/4195595439 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5cffff640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d80c5610 0x7fc5d80ab6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d77fe640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d80c8b90 0x7fc5d80c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.024 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d80c8b90 msgr2=0x7fc5d80c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d80c8b90 0x7fc5d80c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d80c5610 msgr2=0x7fc5d80ab6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d80c5610 0x7fc5d80ab6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 -- 172.21.15.160:0/4195595439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d80ac670 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5cffff640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d80c5610 0x7fc5d80ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5d6ffd640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 0x7fc5d80c48b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8035040 tx=0x7fc5c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8037050 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5c8038040 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004470 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d81ebe60 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.010+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d81eb5c0 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.011+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d8069150 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.012+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fc5c802ac90 con 0x7fc5d80be320 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.012+0000 7fc5cf7fe640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc5b806be30 0x7fc5b806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.025 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.012+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fc5c80abae0 con 0x7fc5d80be320 2024-05-09T07:39:43.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.012+0000 7fc5cffff640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc5b806be30 0x7fc5b806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.026 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.012+0000 7fc5cffff640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc5b806be30 0x7fc5b806e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80cbc80 tx=0x7fc5c0008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.028 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.016+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc5d8069150 con 0x7fc5d80be320 2024-05-09T07:39:43.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.194+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fc5d80756c0 con 0x7fc5b806be30 2024-05-09T07:39:43.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.195+0000 7fc5cf7fe640 1 -- 172.21.15.160:0/4195595439 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5d80756c0 con 0x7fc5b806be30 2024-05-09T07:39:43.207 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 5 to deep-scrub 2024-05-09T07:39:43.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc5b806be30 msgr2=0x7fc5b806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc5b806be30 0x7fc5b806e2f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fc5d80cbc80 tx=0x7fc5c0008040 comp rx=0 tx=0).stop 2024-05-09T07:39:43.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 msgr2=0x7fc5d80c48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 0x7fc5d80c48b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8035040 tx=0x7fc5c80046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 shutdown_connections 2024-05-09T07:39:43.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fc5b806be30 0x7fc5b806e2f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fc5d80c5610 0x7fc5d80ab6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fc5d80c8b90 0x7fc5d80c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 --2- 172.21.15.160:0/4195595439 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc5d80be320 0x7fc5d80c48b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 >> 172.21.15.160:0/4195595439 conn(0x7fc5d8072e90 msgr2=0x7fc5d81516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 shutdown_connections 2024-05-09T07:39:43.211 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.197+0000 7fc5dd3b7640 1 -- 172.21.15.160:0/4195595439 wait complete. 2024-05-09T07:39:43.220 INFO:tasks.ceph:Scrubbing osd.6 2024-05-09T07:39:43.221 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:43.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.317+0000 7f3b2939d640 1 Processor -- start 2024-05-09T07:39:43.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 -- start start 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b24073490 0x7f3b24155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f3b240c1fb0 0x7f3b2406b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b240c07d0 0x7f3b240c0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f3b24155a20 con 0x7f3b240c07d0 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f3b24155440 con 0x7f3b240c1fb0 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.318+0000 7f3b2939d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3b240c1400 con 0x7f3b24073490 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b240c07d0 0x7f3b240c0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f3b240c07d0 0x7f3b240c0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48300/0 (socket says 172.21.15.160:48300) 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 -- 172.21.15.160:0/3393718730 learned_addr learned my addr 172.21.15.160:0/3393718730 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b22ffd640 1 --2- 172.21.15.160:0/3393718730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b24073490 0x7f3b24155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b227fc640 1 --2- 172.21.15.160:0/3393718730 >> v2:172.21.15.188:3300/0 conn(0x7f3b240c1fb0 0x7f3b2406b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 -- 172.21.15.160:0/3393718730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b24073490 msgr2=0x7f3b24155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 --2- 172.21.15.160:0/3393718730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b24073490 0x7f3b24155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 -- 172.21.15.160:0/3393718730 >> v2:172.21.15.188:3300/0 conn(0x7f3b240c1fb0 msgr2=0x7f3b2406b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 --2- 172.21.15.160:0/3393718730 >> v2:172.21.15.188:3300/0 conn(0x7f3b240c1fb0 0x7f3b2406b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b237fe640 1 -- 172.21.15.160:0/3393718730 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b240c16e0 con 0x7f3b240c07d0 2024-05-09T07:39:43.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.319+0000 7f3b227fc640 1 --2- 172.21.15.160:0/3393718730 >> v2:172.21.15.188:3300/0 conn(0x7f3b240c1fb0 0x7f3b2406b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:39:43.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.320+0000 7f3b237fe640 1 --2- 172.21.15.160:0/3393718730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b240c07d0 0x7f3b240c0c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b24070750 tx=0x7f3b1c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d823a6af389d6b6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.320+0000 7f3b21ffb640 1 -- 172.21.15.160:0/3393718730 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c036070 con 0x7f3b240c07d0 2024-05-09T07:39:43.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.320+0000 7f3b21ffb640 1 -- 172.21.15.160:0/3393718730 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1c031600 con 0x7f3b240c07d0 2024-05-09T07:39:43.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.320+0000 7f3b21ffb640 1 -- 172.21.15.160:0/3393718730 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0319d0 con 0x7f3b240c07d0 2024-05-09T07:39:43.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 -- 172.21.15.160:0/3393718730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b240c07d0 msgr2=0x7f3b240c0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 --2- 172.21.15.160:0/3393718730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b240c07d0 0x7f3b240c0c30 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b24070750 tx=0x7f3b1c02a610 comp rx=0 tx=0).stop 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 -- 172.21.15.160:0/3393718730 shutdown_connections 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 --2- 172.21.15.160:0/3393718730 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b240c07d0 0x7f3b240c0c30 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 --2- 172.21.15.160:0/3393718730 >> v2:172.21.15.188:3300/0 conn(0x7f3b240c1fb0 0x7f3b2406b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 --2- 172.21.15.160:0/3393718730 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b24073490 0x7f3b24155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 -- 172.21.15.160:0/3393718730 >> 172.21.15.160:0/3393718730 conn(0x7f3b240c3ea0 msgr2=0x7f3b240c42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 -- 172.21.15.160:0/3393718730 shutdown_connections 2024-05-09T07:39:43.334 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.321+0000 7f3b2939d640 1 -- 172.21.15.160:0/3393718730 wait complete. 2024-05-09T07:39:43.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.323+0000 7f3b2939d640 1 Processor -- start 2024-05-09T07:39:43.335 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.323+0000 7f3b2939d640 1 -- start start 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b2939d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24073490 0x7f3b24152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b2939d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b240c07d0 0x7f3b240c6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b2939d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b240c1fb0 0x7f3b24148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b2939d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f3b240c19c0 con 0x7f3b24073490 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b2939d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f3b24067560 con 0x7f3b240c07d0 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b2939d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f3b24066f90 con 0x7f3b240c1fb0 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24073490 0x7f3b24152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24073490 0x7f3b24152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48306/0 (socket says 172.21.15.160:48306) 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 -- 172.21.15.160:0/2907059588 learned_addr learned my addr 172.21.15.160:0/2907059588 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b237fe640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b240c1fb0 0x7f3b24148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b227fc640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b240c07d0 0x7f3b240c6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 -- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b240c1fb0 msgr2=0x7f3b24148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.337 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b240c1fb0 0x7f3b24148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 -- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b240c07d0 msgr2=0x7f3b240c6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b240c07d0 0x7f3b240c6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b22ffd640 1 -- 172.21.15.160:0/2907059588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b241531a0 con 0x7f3b24073490 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b227fc640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f3b240c07d0 0x7f3b240c6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.324+0000 7f3b237fe640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f3b240c1fb0 0x7f3b24148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.325+0000 7f3b22ffd640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24073490 0x7f3b24152c60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e9e0 tx=0x7f3b1400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.325+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b14017070 con 0x7f3b24073490 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.325+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b1400c9d0 con 0x7f3b24073490 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.325+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b241584b0 con 0x7f3b24073490 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.325+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1400cdc0 con 0x7f3b24073490 2024-05-09T07:39:43.338 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.325+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b24158a70 con 0x7f3b24073490 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.326+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b241596f0 con 0x7f3b24073490 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.327+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f3b1401a070 con 0x7f3b24073490 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.327+0000 7f3b0b7fe640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3b1006be60 0x7f3b1006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.327+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f3b1408b8f0 con 0x7f3b24073490 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.327+0000 7f3b227fc640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3b1006be60 0x7f3b1006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.328+0000 7f3b0b7fe640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3b10075800 0x7f3b10077ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.328+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 --> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b14013e50 con 0x7f3b10075800 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.328+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3b1408bd80 con 0x7f3b24073490 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.328+0000 7f3b237fe640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3b10075800 0x7f3b10077ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.328+0000 7f3b227fc640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3b1006be60 0x7f3b1006e320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009560 tx=0x7f3b18009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.341 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.329+0000 7f3b237fe640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3b10075800 0x7f3b10077ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.342 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.330+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== osd.6 v2:172.21.15.188:6824/566542668 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b14013e50 con 0x7f3b10075800 2024-05-09T07:39:43.360 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.349+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 --> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3b241596f0 con 0x7f3b10075800 2024-05-09T07:39:43.361 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.349+0000 7f3b0b7fe640 1 -- 172.21.15.160:0/2907059588 <== osd.6 v2:172.21.15.188:6824/566542668 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3b241596f0 con 0x7f3b10075800 2024-05-09T07:39:43.361 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:43.361 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:43.361 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:43.362 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3b10075800 msgr2=0x7f3b10077ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.188:6824/566542668,v1:172.21.15.188:6825/566542668] conn(0x7f3b10075800 0x7f3b10077ca0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3b1006be60 msgr2=0x7f3b1006e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f3b1006be60 0x7f3b1006e320 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009560 tx=0x7f3b18009190 comp rx=0 tx=0).stop 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24073490 msgr2=0x7f3b24152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 --2- 172.21.15.160:0/2907059588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b24073490 0x7f3b24152c60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400e9e0 tx=0x7f3b1400eea0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b237fe640 1 -- 172.21.15.160:0/2907059588 reap_dead start 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 shutdown_connections 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 >> 172.21.15.160:0/2907059588 conn(0x7f3b240c3ea0 msgr2=0x7f3b2406b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 shutdown_connections 2024-05-09T07:39:43.363 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.350+0000 7f3b2939d640 1 -- 172.21.15.160:0/2907059588 wait complete. 2024-05-09T07:39:43.372 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-09T07:39:43.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.464+0000 7efd2a509640 1 Processor -- start 2024-05-09T07:39:43.476 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.464+0000 7efd2a509640 1 -- start start 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd2a509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd2a509640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd240c8b90 0x7efd24155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd2a509640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd24159840 con 0x7efd240c5610 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd2a509640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7efd24159310 con 0x7efd240c8b90 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd2a509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd24159560 con 0x7efd240be320 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd28aa6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd240c8b90 0x7efd24155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd23fff640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7efd240c5610 0x7efd24155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:59682/0 (socket says 172.21.15.160:59682) 2024-05-09T07:39:43.478 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd23fff640 1 -- 172.21.15.160:0/576622223 learned_addr learned my addr 172.21.15.160:0/576622223 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd237fe640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd28aa6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7efd240c8b90 0x7efd24155a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.188:3300/0 says I am v2:172.21.15.160:35558/0 (socket says 172.21.15.160:35558) 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd28aa6640 1 -- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 msgr2=0x7efd240c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd28aa6640 1 --2- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd28aa6640 1 -- 172.21.15.160:0/576622223 >> v1:172.21.15.188:6789/0 conn(0x7efd240c5610 legacy=0x7efd24155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd28aa6640 1 -- 172.21.15.160:0/576622223 --> v2:172.21.15.188:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd240c77d0 con 0x7efd240c8b90 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.465+0000 7efd22ffd640 1 -- 172.21.15.160:0/576622223 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4088222175 0 0) 0x7efd24159840 con 0x7efd240c5610 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.466+0000 7efd28aa6640 1 --2- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd24155a20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd140097a0 tx=0x7efd1402a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a56cc670faffe68a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.466+0000 7efd22ffd640 1 -- 172.21.15.160:0/576622223 <== mon.1 v2:172.21.15.188:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14037070 con 0x7efd240c8b90 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.466+0000 7efd22ffd640 1 -- 172.21.15.160:0/576622223 <== mon.1 v2:172.21.15.188:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1402adb0 con 0x7efd240c8b90 2024-05-09T07:39:43.479 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.466+0000 7efd22ffd640 1 -- 172.21.15.160:0/576622223 <== mon.1 v2:172.21.15.188:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd14032660 con 0x7efd240c8b90 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 -- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 msgr2=0x7efd24155a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 --2- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd24155a20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7efd140097a0 tx=0x7efd1402a060 comp rx=0 tx=0).stop 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 -- 172.21.15.160:0/576622223 shutdown_connections 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 --2- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd24155a20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 --2- 172.21.15.160:0/576622223 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240c59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 -- 172.21.15.160:0/576622223 >> 172.21.15.160:0/576622223 conn(0x7efd24072e90 msgr2=0x7efd240ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 -- 172.21.15.160:0/576622223 shutdown_connections 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.467+0000 7efd2a509640 1 -- 172.21.15.160:0/576622223 wait complete. 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.468+0000 7efd2a509640 1 Processor -- start 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.468+0000 7efd2a509640 1 -- start start 2024-05-09T07:39:43.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd2a509640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240bea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd2a509640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 0x7efd2414be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd2a509640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd2414c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd2a509640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efd240ad6a0 con 0x7efd240c5610 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd2a509640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efd240c7d90 con 0x7efd240c8b90 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd2a509640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efd24155f60 con 0x7efd240be320 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 0x7efd2414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 0x7efd2414be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48308/0 (socket says 172.21.15.160:48308) 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 -- 172.21.15.160:0/1753859969 learned_addr learned my addr 172.21.15.160:0/1753859969 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd28aa6640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd2414c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd23fff640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 -- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 msgr2=0x7efd240bea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240bea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 -- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 msgr2=0x7efd2414c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd2414c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.483 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.469+0000 7efd237fe640 1 -- 172.21.15.160:0/1753859969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd24157d60 con 0x7efd240c5610 2024-05-09T07:39:43.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.470+0000 7efd237fe640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 0x7efd2414be40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd1800e9b0 tx=0x7efd1800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.470+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd18017070 con 0x7efd240c5610 2024-05-09T07:39:43.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.470+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1800c9a0 con 0x7efd240c5610 2024-05-09T07:39:43.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.470+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd24156e60 con 0x7efd240c5610 2024-05-09T07:39:43.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.470+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1800cd70 con 0x7efd240c5610 2024-05-09T07:39:43.484 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.470+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd24157420 con 0x7efd240c5610 2024-05-09T07:39:43.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.471+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd24069150 con 0x7efd240c5610 2024-05-09T07:39:43.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.471+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efd1801a070 con 0x7efd240c5610 2024-05-09T07:39:43.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.472+0000 7efd20ff9640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efd1006be30 0x7efd1006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.472+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7efd1808b560 con 0x7efd240c5610 2024-05-09T07:39:43.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.472+0000 7efd23fff640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efd1006be30 0x7efd1006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.485 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.472+0000 7efd23fff640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efd1006be30 0x7efd1006e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0012e0 tx=0x7efd1c001700 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.488 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.476+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd24069150 con 0x7efd240c5610 2024-05-09T07:39:43.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.675+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7efd24156c10 con 0x7efd1006be30 2024-05-09T07:39:43.687 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.675+0000 7efd20ff9640 1 -- 172.21.15.160:0/1753859969 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efd24156c10 con 0x7efd1006be30 2024-05-09T07:39:43.687 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 6 to deep-scrub 2024-05-09T07:39:43.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.677+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efd1006be30 msgr2=0x7efd1006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.689 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.677+0000 7efd2a509640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efd1006be30 0x7efd1006e2f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0012e0 tx=0x7efd1c001700 comp rx=0 tx=0).stop 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.677+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 msgr2=0x7efd2414be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.677+0000 7efd2a509640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 0x7efd2414be40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7efd1800e9b0 tx=0x7efd1800ee70 comp rx=0 tx=0).stop 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.677+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 shutdown_connections 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.677+0000 7efd2a509640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efd1006be30 0x7efd1006e2f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.678+0000 7efd2a509640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efd240c8b90 0x7efd2414c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.678+0000 7efd2a509640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efd240c5610 0x7efd2414be40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.678+0000 7efd2a509640 1 --2- 172.21.15.160:0/1753859969 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efd240be320 0x7efd240bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.678+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 >> 172.21.15.160:0/1753859969 conn(0x7efd24072e90 msgr2=0x7efd2414e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.678+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 shutdown_connections 2024-05-09T07:39:43.690 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.678+0000 7efd2a509640 1 -- 172.21.15.160:0/1753859969 wait complete. 2024-05-09T07:39:43.701 INFO:tasks.ceph:Scrubbing osd.7 2024-05-09T07:39:43.701 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-09T07:39:43.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.800+0000 7f25ac712640 1 Processor -- start 2024-05-09T07:39:43.812 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.800+0000 7f25ac712640 1 -- start start 2024-05-09T07:39:43.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25ac712640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a4159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25ac712640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f25a40c5690 0x7f25a40c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25ac712640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f25a4156d90 con 0x7f25a4152a80 2024-05-09T07:39:43.813 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25ac712640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f25a4152e60 con 0x7f25a40c5690 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25ac712640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f25a40c80a0 con 0x7f25a41568e0 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25aa4ae640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f25a4152a80 0x7f25a4155e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51934/0 (socket says 172.21.15.160:51934) 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25aa4ae640 1 -- 172.21.15.160:0/464747305 learned_addr learned my addr 172.21.15.160:0/464747305 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.801+0000 7f25aacaf640 1 --2- 172.21.15.160:0/464747305 >> v2:172.21.15.160:3300/0 conn(0x7f25a40c5690 0x7f25a40c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.802+0000 7f25aacaf640 1 -- 172.21.15.160:0/464747305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 msgr2=0x7f25a4159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.802+0000 7f25aacaf640 1 --2- 172.21.15.160:0/464747305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a4159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.802+0000 7f25aacaf640 1 -- 172.21.15.160:0/464747305 >> v1:172.21.15.160:6789/0 conn(0x7f25a4152a80 legacy=0x7f25a4155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.802+0000 7f25a94ac640 1 -- 172.21.15.160:0/464747305 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3460905541 0 0) 0x7f25a4156d90 con 0x7f25a4152a80 2024-05-09T07:39:43.814 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.802+0000 7f25aacaf640 1 -- 172.21.15.160:0/464747305 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a40c7ae0 con 0x7f25a40c5690 2024-05-09T07:39:43.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.803+0000 7f25aacaf640 1 --2- 172.21.15.160:0/464747305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a40c5690 0x7f25a40c5ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d90 tx=0x7f259c02a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ea49a53b68fd56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.803+0000 7f25a94ac640 1 -- 172.21.15.160:0/464747305 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c007740 con 0x7f25a40c5690 2024-05-09T07:39:43.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.803+0000 7f25a94ac640 1 -- 172.21.15.160:0/464747305 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f259c0078e0 con 0x7f25a40c5690 2024-05-09T07:39:43.815 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.803+0000 7f25a94ac640 1 -- 172.21.15.160:0/464747305 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f259c007cb0 con 0x7f25a40c5690 2024-05-09T07:39:43.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 -- 172.21.15.160:0/464747305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a40c5690 msgr2=0x7f25a40c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 --2- 172.21.15.160:0/464747305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a40c5690 0x7f25a40c5ae0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f259c009d90 tx=0x7f259c02a530 comp rx=0 tx=0).stop 2024-05-09T07:39:43.816 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 -- 172.21.15.160:0/464747305 shutdown_connections 2024-05-09T07:39:43.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 --2- 172.21.15.160:0/464747305 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a40c5690 0x7f25a40c5ae0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 --2- 172.21.15.160:0/464747305 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a4159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 -- 172.21.15.160:0/464747305 >> 172.21.15.160:0/464747305 conn(0x7f25a4072ec0 msgr2=0x7f25a40ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 -- 172.21.15.160:0/464747305 shutdown_connections 2024-05-09T07:39:43.817 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.804+0000 7f25ac712640 1 -- 172.21.15.160:0/464747305 wait complete. 2024-05-09T07:39:43.819 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.805+0000 7f25ac712640 1 Processor -- start 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.805+0000 7f25ac712640 1 -- start start 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25ac712640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a40c5690 0x7f25a414d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25ac712640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a4152a80 0x7f25a41eab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25ac712640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a414a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25ac712640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4158df0 con 0x7f25a4152a80 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25ac712640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f25a4158b10 con 0x7f25a40c5690 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25ac712640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f25a40ad6a0 con 0x7f25a41568e0 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25a9cad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a4152a80 0x7f25a41eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25a9cad640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a4152a80 0x7f25a41eab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48320/0 (socket says 172.21.15.160:48320) 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.806+0000 7f25a9cad640 1 -- 172.21.15.160:0/2836524178 learned_addr learned my addr 172.21.15.160:0/2836524178 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:43.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25aa4ae640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a40c5690 0x7f25a414d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25aacaf640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a414a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25a9cad640 1 -- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 msgr2=0x7f25a414a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25a9cad640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a414a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25a9cad640 1 -- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a40c5690 msgr2=0x7f25a414d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25a9cad640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a40c5690 0x7f25a414d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25a9cad640 1 -- 172.21.15.160:0/2836524178 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25a414cf30 con 0x7f25a4152a80 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25aa4ae640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f25a40c5690 0x7f25a414d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25aacaf640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f25a41568e0 0x7f25a414a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25a9cad640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a4152a80 0x7f25a41eab80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f259400b4a0 tx=0x7f259400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594014070 con 0x7f25a4152a80 2024-05-09T07:39:43.821 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2594010610 con 0x7f25a4152a80 2024-05-09T07:39:43.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25a40c0130 con 0x7f25a4152a80 2024-05-09T07:39:43.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2594010a00 con 0x7f25a4152a80 2024-05-09T07:39:43.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.807+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25a40c5240 con 0x7f25a4152a80 2024-05-09T07:39:43.822 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.808+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f25a40bff20 con 0x7f25a4152a80 2024-05-09T07:39:43.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.809+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f2594017070 con 0x7f25a4152a80 2024-05-09T07:39:43.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.809+0000 7f259affd640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.809+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f259408acd0 con 0x7f25a4152a80 2024-05-09T07:39:43.823 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.809+0000 7f25aa4ae640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f258806be30 0x7f258806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.810+0000 7f259affd640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7f2588075740 0x7f2588077be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.810+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 --> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2594011ec0 con 0x7f2588075740 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.810+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f259408d3d0 con 0x7f25a4152a80 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.810+0000 7f25aacaf640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7f2588075740 0x7f2588077be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.810+0000 7f25aa4ae640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25a0000f60 tx=0x7f25a0007c30 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.811+0000 7f25aacaf640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7f2588075740 0x7f2588077be0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.824 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.811+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== osd.7 v2:172.21.15.188:6808/4113254918 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2594011ec0 con 0x7f2588075740 2024-05-09T07:39:43.839 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.828+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 --> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f25a414fbd0 con 0x7f2588075740 2024-05-09T07:39:43.840 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.828+0000 7f259affd640 1 -- 172.21.15.160:0/2836524178 <== osd.7 v2:172.21.15.188:6808/4113254918 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f25a414fbd0 con 0x7f2588075740 2024-05-09T07:39:43.840 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2024-05-09T07:39:43.841 INFO:teuthology.orchestra.run.smithi160.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-09T07:39:43.841 INFO:teuthology.orchestra.run.smithi160.stdout:} 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7f2588075740 msgr2=0x7f2588077be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.188:6808/4113254918,v1:172.21.15.188:6809/4113254918] conn(0x7f2588075740 0x7f2588077be0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f258806be30 msgr2=0x7f258806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f258806be30 0x7f258806e2f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25a0000f60 tx=0x7f25a0007c30 comp rx=0 tx=0).stop 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a4152a80 msgr2=0x7f25a41eab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 --2- 172.21.15.160:0/2836524178 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25a4152a80 0x7f25a41eab80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f259400b4a0 tx=0x7f259400b960 comp rx=0 tx=0).stop 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25aacaf640 1 -- 172.21.15.160:0/2836524178 reap_dead start 2024-05-09T07:39:43.842 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 shutdown_connections 2024-05-09T07:39:43.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 >> 172.21.15.160:0/2836524178 conn(0x7f25a4072ec0 msgr2=0x7f25a4156130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 shutdown_connections 2024-05-09T07:39:43.843 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.829+0000 7f25ac712640 1 -- 172.21.15.160:0/2836524178 wait complete. 2024-05-09T07:39:43.854 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-09T07:39:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.947+0000 7efc6964d640 1 Processor -- start 2024-05-09T07:39:43.959 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.947+0000 7efc6964d640 1 -- start start 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc6964d640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efc640c8bc0 0x7efc64155f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc6964d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64159450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc6964d640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efc640c7870 con 0x7efc64152a30 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc6964d640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7efc640c7b50 con 0x7efc640c8bc0 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc6964d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efc640c7e30 con 0x7efc64159020 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc5bfff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7efc640c8bc0 0x7efc64155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc62ffd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7efc64152a30 0x7efc640c5640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:51938/0 (socket says 172.21.15.160:51938) 2024-05-09T07:39:43.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc62ffd640 1 -- 172.21.15.160:0/2820961076 learned_addr learned my addr 172.21.15.160:0/2820961076 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc637fe640 1 --2- 172.21.15.160:0/2820961076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc5bfff640 1 -- 172.21.15.160:0/2820961076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 msgr2=0x7efc64159450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc5bfff640 1 --2- 172.21.15.160:0/2820961076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc5bfff640 1 -- 172.21.15.160:0/2820961076 >> v1:172.21.15.160:6789/0 conn(0x7efc64152a30 legacy=0x7efc640c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc5bfff640 1 -- 172.21.15.160:0/2820961076 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc640c8110 con 0x7efc640c8bc0 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.948+0000 7efc627fc640 1 -- 172.21.15.160:0/2820961076 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1418835160 0 0) 0x7efc640c7870 con 0x7efc64152a30 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.949+0000 7efc5bfff640 1 --2- 172.21.15.160:0/2820961076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc640c8bc0 0x7efc64155f10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009790 tx=0x7efc4c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86abffa3718ae572 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc627fc640 1 -- 172.21.15.160:0/2820961076 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c037070 con 0x7efc640c8bc0 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc627fc640 1 -- 172.21.15.160:0/2820961076 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c032440 con 0x7efc640c8bc0 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc627fc640 1 -- 172.21.15.160:0/2820961076 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c032810 con 0x7efc640c8bc0 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 -- 172.21.15.160:0/2820961076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc640c8bc0 msgr2=0x7efc64155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 --2- 172.21.15.160:0/2820961076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc640c8bc0 0x7efc64155f10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc4c009790 tx=0x7efc4c02a180 comp rx=0 tx=0).stop 2024-05-09T07:39:43.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 -- 172.21.15.160:0/2820961076 shutdown_connections 2024-05-09T07:39:43.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 --2- 172.21.15.160:0/2820961076 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64159450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 --2- 172.21.15.160:0/2820961076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc640c8bc0 0x7efc64155f10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 -- 172.21.15.160:0/2820961076 >> 172.21.15.160:0/2820961076 conn(0x7efc64072ec0 msgr2=0x7efc640ac870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:43.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 -- 172.21.15.160:0/2820961076 shutdown_connections 2024-05-09T07:39:43.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.950+0000 7efc6964d640 1 -- 172.21.15.160:0/2820961076 wait complete. 2024-05-09T07:39:43.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.951+0000 7efc6964d640 1 Processor -- start 2024-05-09T07:39:43.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.951+0000 7efc6964d640 1 -- start start 2024-05-09T07:39:43.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc6964d640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc640c8bc0 0x7efc640ac610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc6964d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 0x7efc64149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc6964d640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc6964d640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7efc640c5d50 con 0x7efc64152a30 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc6964d640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7efc64152e10 con 0x7efc640c8bc0 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc6964d640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7efc640c5ed0 con 0x7efc64159020 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 0x7efc64149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 0x7efc64149a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48336/0 (socket says 172.21.15.160:48336) 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 -- 172.21.15.160:0/899729236 learned_addr learned my addr 172.21.15.160:0/899729236 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc637fe640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc62ffd640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc640c8bc0 0x7efc640ac610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 -- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 msgr2=0x7efc64153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.967 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 -- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc640c8bc0 msgr2=0x7efc640ac610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc640c8bc0 0x7efc640ac610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc5bfff640 1 -- 172.21.15.160:0/899729236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc640abfa0 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.952+0000 7efc62ffd640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc640c8bc0 0x7efc640ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.953+0000 7efc5bfff640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 0x7efc64149a70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098b0 tx=0x7efc4c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.953+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c037040 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.953+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc4c005b90 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.953+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc640aa9c0 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.953+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc4c030770 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.953+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc640abcc0 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.954+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc640756c0 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.954+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7efc4c030910 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.955+0000 7efc5b7fe640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efc4406be30 0x7efc4406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.955+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7efc4c0abc70 con 0x7efc64152a30 2024-05-09T07:39:43.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.955+0000 7efc62ffd640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efc4406be30 0x7efc4406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:43.969 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.955+0000 7efc62ffd640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efc4406be30 0x7efc4406e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc54000f60 tx=0x7efc54009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:43.974 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:43.963+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efc4c032e20 con 0x7efc64152a30 2024-05-09T07:39:44.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.146+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7efc640c8350 con 0x7efc4406be30 2024-05-09T07:39:44.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.147+0000 7efc5b7fe640 1 -- 172.21.15.160:0/899729236 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efc640c8350 con 0x7efc4406be30 2024-05-09T07:39:44.159 INFO:teuthology.orchestra.run.smithi160.stderr:instructed osd(s) 7 to deep-scrub 2024-05-09T07:39:44.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efc4406be30 msgr2=0x7efc4406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efc4406be30 0x7efc4406e2f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7efc54000f60 tx=0x7efc54009040 comp rx=0 tx=0).stop 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 msgr2=0x7efc64149a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 0x7efc64149a70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efc4c0098b0 tx=0x7efc4c0046c0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 shutdown_connections 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7efc4406be30 0x7efc4406e2f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7efc64159020 0x7efc64153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efc64152a30 0x7efc64149a70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 --2- 172.21.15.160:0/899729236 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7efc640c8bc0 0x7efc640ac610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 >> 172.21.15.160:0/899729236 conn(0x7efc64072ec0 msgr2=0x7efc6406c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 shutdown_connections 2024-05-09T07:39:44.163 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.149+0000 7efc6964d640 1 -- 172.21.15.160:0/899729236 wait complete. 2024-05-09T07:39:44.172 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:39:44.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.265+0000 7f1fbc7a2640 1 Processor -- start 2024-05-09T07:39:44.277 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.265+0000 7f1fbc7a2640 1 -- start start 2024-05-09T07:39:44.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fbc7a2640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1fb40c5610 0x7f1fb40c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:44.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fbc7a2640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f1fb40be320 0x7f1fb4155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:44.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fbc7a2640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1fb4159840 con 0x7f1fb40c8b90 2024-05-09T07:39:44.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fbc7a2640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb4159310 con 0x7f1fb40c5610 2024-05-09T07:39:44.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fbc7a2640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f1fb4159560 con 0x7f1fb40be320 2024-05-09T07:39:44.278 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1fb40c5610 0x7f1fb40c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:44.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f1fb40c5610 0x7f1fb40c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48342/0 (socket says 172.21.15.160:48342) 2024-05-09T07:39:44.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 -- 172.21.15.160:0/4285200676 learned_addr learned my addr 172.21.15.160:0/4285200676 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:44.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 -- 172.21.15.160:0/4285200676 >> v2:172.21.15.188:3300/0 conn(0x7f1fb40be320 msgr2=0x7f1fb4155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:44.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 --2- 172.21.15.160:0/4285200676 >> v2:172.21.15.188:3300/0 conn(0x7f1fb40be320 0x7f1fb4155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 -- 172.21.15.160:0/4285200676 >> v1:172.21.15.188:6789/0 conn(0x7f1fb40c8b90 legacy=0x7f1fb4155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:39:44.279 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.266+0000 7f1fba53e640 1 -- 172.21.15.160:0/4285200676 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb40c77d0 con 0x7f1fb40c5610 2024-05-09T07:39:44.280 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.267+0000 7f1fba53e640 1 --2- 172.21.15.160:0/4285200676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c5610 0x7f1fb40c59f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1fb406d860 tx=0x7f1fa802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b04d129e6713ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.267+0000 7f1fb953c640 1 -- 172.21.15.160:0/4285200676 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8037070 con 0x7f1fb40c5610 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.267+0000 7f1fb953c640 1 -- 172.21.15.160:0/4285200676 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fa802ae20 con 0x7f1fb40c5610 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.267+0000 7f1fb953c640 1 -- 172.21.15.160:0/4285200676 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8032660 con 0x7f1fb40c5610 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/4285200676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c5610 msgr2=0x7f1fb40c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/4285200676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c5610 0x7f1fb40c59f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1fb406d860 tx=0x7f1fa802a110 comp rx=0 tx=0).stop 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/4285200676 shutdown_connections 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/4285200676 >> v2:172.21.15.188:3300/0 conn(0x7f1fb40be320 0x7f1fb4155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/4285200676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c5610 0x7f1fb40c59f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/4285200676 >> 172.21.15.160:0/4285200676 conn(0x7f1fb4072e90 msgr2=0x7f1fb40ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/4285200676 shutdown_connections 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.268+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/4285200676 wait complete. 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.269+0000 7f1fbc7a2640 1 Processor -- start 2024-05-09T07:39:44.281 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.269+0000 7f1fbc7a2640 1 -- start start 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb40be320 0x7f1fb4153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fb40c5610 0x7f1fb41573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 0x7f1fb41ef050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40ad6a0 con 0x7f1fb40c8b90 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb40c7d90 con 0x7f1fb40be320 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb4155f60 con 0x7f1fb40c5610 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 0x7f1fb41ef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 0x7f1fb41ef050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48356/0 (socket says 172.21.15.160:48356) 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 -- 172.21.15.160:0/938306264 learned_addr learned my addr 172.21.15.160:0/938306264 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fba53e640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb40be320 0x7f1fb4153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fb9d3d640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fb40c5610 0x7f1fb41573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 -- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fb40c5610 msgr2=0x7f1fb41573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.284 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fb40c5610 0x7f1fb41573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 -- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb40be320 msgr2=0x7f1fb4153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb40be320 0x7f1fb4153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 -- 172.21.15.160:0/938306264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb41ee890 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbad3f640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 0x7f1fb41ef050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b990 tx=0x7f1fac00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac019070 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1fac00cba0 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb4154b60 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.270+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fac014440 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.271+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41eeb30 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.272+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb40756c0 con 0x7f1fb40c8b90 2024-05-09T07:39:44.285 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.272+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f1fac01c070 con 0x7f1fb40c8b90 2024-05-09T07:39:44.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.273+0000 7f1fa6ffd640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1f9806be30 0x7f1f9806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:39:44.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.273+0000 7f1fba53e640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1f9806be30 0x7f1f9806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:39:44.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.273+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f1fac08af60 con 0x7f1fb40c8b90 2024-05-09T07:39:44.286 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.273+0000 7f1fba53e640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1f9806be30 0x7f1f9806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8034000 tx=0x7f1fa802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:39:44.288 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.277+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1fac0595b0 con 0x7f1fb40c8b90 2024-05-09T07:39:44.482 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.471+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1fb40be100 con 0x7f1f9806be30 2024-05-09T07:39:44.490 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.479+0000 7f1fa6ffd640 1 -- 172.21.15.160:0/938306264 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+335751 (secure 0 0 0) 0x7f1fb40be100 con 0x7f1f9806be30 2024-05-09T07:39:44.494 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:39:44.496 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:39:44.497 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1f9806be30 msgr2=0x7f1f9806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1f9806be30 0x7f1f9806e2f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8034000 tx=0x7f1fa802e000 comp rx=0 tx=0).stop 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 msgr2=0x7f1fb41ef050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 0x7f1fb41ef050 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1fac00b990 tx=0x7f1fac00be50 comp rx=0 tx=0).stop 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 shutdown_connections 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f1f9806be30 0x7f1f9806e2f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1fb40c8b90 0x7f1fb41ef050 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f1fb40c5610 0x7f1fb41573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 --2- 172.21.15.160:0/938306264 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f1fb40be320 0x7f1fb4153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 >> 172.21.15.160:0/938306264 conn(0x7f1fb4072e90 msgr2=0x7f1fb4151ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 shutdown_connections 2024-05-09T07:39:44.498 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:39:44.482+0000 7f1fbc7a2640 1 -- 172.21.15.160:0/938306264 wait complete. 2024-05-09T07:39:44.505 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":805,"stamp":"2024-05-09T07:39:43.562913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":249298558,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":15568,"num_read_kb":3136890,"num_write":427811,"num_write_kb":32420213,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":590023,"ondisk_log_size":590023,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8152856,"kb_used_data":489432,"kb_used_omap":12,"kb_used_meta":7663347,"kb_avail":746821864,"statfs":{"total":773094113280,"available":764745588736,"internally_reserved":0,"allocated":501178368,"data_stored":499756948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":7847267889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-252579696,"num_objects":-1981,"num_object_clones":0,"num_object_copies":-6582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1981,"num_whiteouts":0,"num_read":10,"num_read_kb":139,"num_write":2884,"num_write_kb":32965,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-112,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003163"},"pg_stats":[{"pgid":"2.1f","version":"27'2675","reported_seq":2714,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.220979+0000","last_change":"2024-05-09T07:13:44.474800+0000","last_active":"2024-05-09T07:39:37.220979+0000","last_peered":"2024-05-09T07:39:37.220979+0000","last_clean":"2024-05-09T07:39:37.220979+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:39:37.220979+0000","last_undegraded":"2024-05-09T07:39:37.220979+0000","last_fullsized":"2024-05-09T07:39:37.220979+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2675,"log_dups_size":0,"ondisk_log_size":2675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":2687,"num_write_kb":18241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2578","reported_seq":2616,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.707984+0000","last_change":"2024-05-09T07:13:45.492188+0000","last_active":"2024-05-09T07:39:37.707984+0000","last_peered":"2024-05-09T07:39:37.707984+0000","last_clean":"2024-05-09T07:39:37.707984+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:39:37.707984+0000","last_undegraded":"2024-05-09T07:39:37.707984+0000","last_fullsized":"2024-05-09T07:39:37.707984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9823","reported_seq":10253,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.599216+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:39:34.599216+0000","last_peered":"2024-05-09T07:39:34.599216+0000","last_clean":"2024-05-09T07:39:34.599216+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:39:34.599216+0000","last_undegraded":"2024-05-09T07:39:34.599216+0000","last_fullsized":"2024-05-09T07:39:34.599216+0000","mapping_epoch":22,"log_start":"27'3504","ondisk_log_start":"27'3504","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6319,"log_dups_size":0,"ondisk_log_size":6319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:27.076245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":399,"num_read_kb":101346,"num_write":7176,"num_write_kb":1009567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'7790","reported_seq":7830,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.271128+0000","last_change":"2024-05-09T07:13:44.460701+0000","last_active":"2024-05-09T07:39:37.271128+0000","last_peered":"2024-05-09T07:39:37.271128+0000","last_clean":"2024-05-09T07:39:37.271128+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:39:37.271128+0000","last_undegraded":"2024-05-09T07:39:37.271128+0000","last_fullsized":"2024-05-09T07:39:37.271128+0000","mapping_epoch":17,"log_start":"27'1500","ondisk_log_start":"27'1500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6290,"log_dups_size":0,"ondisk_log_size":6290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:03:56.916984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":7802,"num_write_kb":54453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2558","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.720510+0000","last_change":"2024-05-09T07:13:45.489309+0000","last_active":"2024-05-09T07:39:37.720510+0000","last_peered":"2024-05-09T07:39:37.720510+0000","last_clean":"2024-05-09T07:39:37.720510+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:39:37.720510+0000","last_undegraded":"2024-05-09T07:39:37.720510+0000","last_fullsized":"2024-05-09T07:39:37.720510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9533","reported_seq":9920,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.634817+0000","last_change":"2024-05-09T07:13:49.505859+0000","last_active":"2024-05-09T07:39:34.634817+0000","last_peered":"2024-05-09T07:39:34.634817+0000","last_clean":"2024-05-09T07:39:34.634817+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:39:34.634817+0000","last_undegraded":"2024-05-09T07:39:34.634817+0000","last_fullsized":"2024-05-09T07:39:34.634817+0000","mapping_epoch":22,"log_start":"27'2329","ondisk_log_start":"27'2329","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7204,"log_dups_size":0,"ondisk_log_size":7204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6761,"num_write_kb":1095702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9531","reported_seq":9709,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:38.173673+0000","last_change":"2024-05-09T07:13:44.474349+0000","last_active":"2024-05-09T07:39:38.173673+0000","last_peered":"2024-05-09T07:39:38.173673+0000","last_clean":"2024-05-09T07:39:38.173673+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:39:38.173673+0000","last_undegraded":"2024-05-09T07:39:38.173673+0000","last_fullsized":"2024-05-09T07:39:38.173673+0000","mapping_epoch":17,"log_start":"27'2500","ondisk_log_start":"27'2500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7031,"log_dups_size":0,"ondisk_log_size":7031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5224923,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":1099,"num_write":9529,"num_write_kb":77058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2666","reported_seq":2702,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.713637+0000","last_change":"2024-05-09T07:13:45.493357+0000","last_active":"2024-05-09T07:39:37.713637+0000","last_peered":"2024-05-09T07:39:37.713637+0000","last_clean":"2024-05-09T07:39:37.713637+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:39:37.713637+0000","last_undegraded":"2024-05-09T07:39:37.713637+0000","last_fullsized":"2024-05-09T07:39:37.713637+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2666,"log_dups_size":0,"ondisk_log_size":2666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9072","reported_seq":9481,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.671984+0000","last_change":"2024-05-09T07:13:49.507490+0000","last_active":"2024-05-09T07:39:34.671984+0000","last_peered":"2024-05-09T07:39:34.671984+0000","last_clean":"2024-05-09T07:39:34.671984+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:39:34.671984+0000","last_undegraded":"2024-05-09T07:39:34.671984+0000","last_fullsized":"2024-05-09T07:39:34.671984+0000","mapping_epoch":22,"log_start":"27'1901","ondisk_log_start":"27'1901","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7171,"log_dups_size":0,"ondisk_log_size":7171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6307,"num_write_kb":900735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4293","reported_seq":4330,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.270951+0000","last_change":"2024-05-09T07:13:44.476186+0000","last_active":"2024-05-09T07:39:37.270951+0000","last_peered":"2024-05-09T07:39:37.270951+0000","last_clean":"2024-05-09T07:39:37.270951+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:39:37.270951+0000","last_undegraded":"2024-05-09T07:39:37.270951+0000","last_fullsized":"2024-05-09T07:39:37.270951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4293,"log_dups_size":0,"ondisk_log_size":4293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":4306,"num_write_kb":47774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2602","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.715829+0000","last_change":"2024-05-09T07:13:45.487573+0000","last_active":"2024-05-09T07:39:37.715829+0000","last_peered":"2024-05-09T07:39:37.715829+0000","last_clean":"2024-05-09T07:39:37.715829+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:39:37.715829+0000","last_undegraded":"2024-05-09T07:39:37.715829+0000","last_fullsized":"2024-05-09T07:39:37.715829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2602,"log_dups_size":0,"ondisk_log_size":2602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'9250","reported_seq":9723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.501486+0000","last_change":"2024-05-09T07:13:49.506989+0000","last_active":"2024-05-09T07:39:34.501486+0000","last_peered":"2024-05-09T07:39:34.501486+0000","last_clean":"2024-05-09T07:39:34.501486+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:39:34.501486+0000","last_undegraded":"2024-05-09T07:39:34.501486+0000","last_fullsized":"2024-05-09T07:39:34.501486+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.067906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":442,"num_read_kb":112268,"num_write":6539,"num_write_kb":853848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2585","reported_seq":2622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.638734+0000","last_change":"2024-05-09T07:13:45.487281+0000","last_active":"2024-05-09T07:39:37.638734+0000","last_peered":"2024-05-09T07:39:37.638734+0000","last_clean":"2024-05-09T07:39:37.638734+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:39:37.638734+0000","last_undegraded":"2024-05-09T07:39:37.638734+0000","last_fullsized":"2024-05-09T07:39:37.638734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6966","reported_seq":7004,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.271727+0000","last_change":"2024-05-09T07:13:44.474398+0000","last_active":"2024-05-09T07:39:37.271727+0000","last_peered":"2024-05-09T07:39:37.271727+0000","last_clean":"2024-05-09T07:39:37.271727+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:39:37.271727+0000","last_undegraded":"2024-05-09T07:39:37.271727+0000","last_fullsized":"2024-05-09T07:39:37.271727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6966,"log_dups_size":0,"ondisk_log_size":6966,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:04.251642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6974,"num_write_kb":53856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9808","reported_seq":10055,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.394694+0000","last_change":"2024-05-09T07:13:49.505753+0000","last_active":"2024-05-09T07:39:34.394694+0000","last_peered":"2024-05-09T07:39:34.394694+0000","last_clean":"2024-05-09T07:39:34.394694+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:39:34.394694+0000","last_undegraded":"2024-05-09T07:39:34.394694+0000","last_fullsized":"2024-05-09T07:39:34.394694+0000","mapping_epoch":22,"log_start":"27'2633","ondisk_log_start":"27'2633","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":215,"num_read_kb":54610,"num_write":7041,"num_write_kb":1008697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2616","reported_seq":2653,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.693543+0000","last_change":"2024-05-09T07:13:45.478948+0000","last_active":"2024-05-09T07:39:37.693543+0000","last_peered":"2024-05-09T07:39:37.693543+0000","last_clean":"2024-05-09T07:39:37.693543+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:39:37.693543+0000","last_undegraded":"2024-05-09T07:39:37.693543+0000","last_fullsized":"2024-05-09T07:39:37.693543+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2616,"log_dups_size":0,"ondisk_log_size":2616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7782","reported_seq":7821,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.270716+0000","last_change":"2024-05-09T07:13:44.469640+0000","last_active":"2024-05-09T07:39:37.270716+0000","last_peered":"2024-05-09T07:39:37.270716+0000","last_clean":"2024-05-09T07:39:37.270716+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:39:37.270716+0000","last_undegraded":"2024-05-09T07:39:37.270716+0000","last_fullsized":"2024-05-09T07:39:37.270716+0000","mapping_epoch":17,"log_start":"27'400","ondisk_log_start":"27'400","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":58435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9632","reported_seq":10172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.578729+0000","last_change":"2024-05-09T07:13:49.508596+0000","last_active":"2024-05-09T07:39:34.578729+0000","last_peered":"2024-05-09T07:39:34.578729+0000","last_clean":"2024-05-09T07:39:34.578729+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:39:34.578729+0000","last_undegraded":"2024-05-09T07:39:34.578729+0000","last_fullsized":"2024-05-09T07:39:34.578729+0000","mapping_epoch":22,"log_start":"27'2443","ondisk_log_start":"27'2443","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":508,"num_read_kb":129802,"num_write":6852,"num_write_kb":970185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2685","reported_seq":2722,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.700443+0000","last_change":"2024-05-09T07:13:45.479787+0000","last_active":"2024-05-09T07:39:37.700443+0000","last_peered":"2024-05-09T07:39:37.700443+0000","last_clean":"2024-05-09T07:39:37.700443+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:39:37.700443+0000","last_undegraded":"2024-05-09T07:39:37.700443+0000","last_fullsized":"2024-05-09T07:39:37.700443+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3775","reported_seq":3814,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:58.748130+0000","last_change":"2024-05-09T07:13:44.475813+0000","last_active":"2024-05-09T07:38:58.748130+0000","last_peered":"2024-05-09T07:38:58.748130+0000","last_clean":"2024-05-09T07:38:58.748130+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:38:58.748130+0000","last_undegraded":"2024-05-09T07:38:58.748130+0000","last_fullsized":"2024-05-09T07:38:58.748130+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3775,"log_dups_size":0,"ondisk_log_size":3775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3772,"num_write_kb":27117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9053","reported_seq":9665,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.462264+0000","last_change":"2024-05-09T07:14:26.646797+0000","last_active":"2024-05-09T07:39:34.462264+0000","last_peered":"2024-05-09T07:39:34.462264+0000","last_clean":"2024-05-09T07:39:34.462264+0000","last_became_active":"2024-05-09T07:14:26.646222+0000","last_became_peered":"2024-05-09T07:14:26.646222+0000","last_unstale":"2024-05-09T07:39:34.462264+0000","last_undegraded":"2024-05-09T07:39:34.462264+0000","last_fullsized":"2024-05-09T07:39:34.462264+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6377,"num_write_kb":893211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2588","reported_seq":2624,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.687209+0000","last_change":"2024-05-09T07:13:45.481727+0000","last_active":"2024-05-09T07:39:37.687209+0000","last_peered":"2024-05-09T07:39:37.687209+0000","last_clean":"2024-05-09T07:39:37.687209+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:39:37.687209+0000","last_undegraded":"2024-05-09T07:39:37.687209+0000","last_fullsized":"2024-05-09T07:39:37.687209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:58.029404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2267","reported_seq":2305,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.230330+0000","last_change":"2024-05-09T07:13:44.475644+0000","last_active":"2024-05-09T07:39:37.230330+0000","last_peered":"2024-05-09T07:39:37.230330+0000","last_clean":"2024-05-09T07:39:37.230330+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:39:37.230330+0000","last_undegraded":"2024-05-09T07:39:37.230330+0000","last_fullsized":"2024-05-09T07:39:37.230330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2267,"log_dups_size":0,"ondisk_log_size":2267,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:53.838218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2276,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9931","reported_seq":10067,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.522470+0000","last_change":"2024-05-09T07:13:49.508430+0000","last_active":"2024-05-09T07:39:34.522470+0000","last_peered":"2024-05-09T07:39:34.522470+0000","last_clean":"2024-05-09T07:39:34.522470+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:39:34.522470+0000","last_undegraded":"2024-05-09T07:39:34.522470+0000","last_fullsized":"2024-05-09T07:39:34.522470+0000","mapping_epoch":22,"log_start":"27'2728","ondisk_log_start":"27'2728","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":26416,"num_write":7236,"num_write_kb":974347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2542","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.681548+0000","last_change":"2024-05-09T07:13:45.477906+0000","last_active":"2024-05-09T07:39:37.681548+0000","last_peered":"2024-05-09T07:39:37.681548+0000","last_clean":"2024-05-09T07:39:37.681548+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:39:37.681548+0000","last_undegraded":"2024-05-09T07:39:37.681548+0000","last_fullsized":"2024-05-09T07:39:37.681548+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4066","reported_seq":4105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:38:39.865043+0000","last_change":"2024-05-09T07:13:44.474570+0000","last_active":"2024-05-09T07:38:39.865043+0000","last_peered":"2024-05-09T07:38:39.865043+0000","last_clean":"2024-05-09T07:38:39.865043+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:38:39.865043+0000","last_undegraded":"2024-05-09T07:38:39.865043+0000","last_fullsized":"2024-05-09T07:38:39.865043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4066,"log_dups_size":0,"ondisk_log_size":4066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":39474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9249","reported_seq":9667,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.503494+0000","last_change":"2024-05-09T07:13:49.941437+0000","last_active":"2024-05-09T07:39:34.503494+0000","last_peered":"2024-05-09T07:39:34.503494+0000","last_clean":"2024-05-09T07:39:34.503494+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:39:34.503494+0000","last_undegraded":"2024-05-09T07:39:34.503494+0000","last_fullsized":"2024-05-09T07:39:34.503494+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7227,"log_dups_size":0,"ondisk_log_size":7227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":97790,"num_write":6537,"num_write_kb":843792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2541","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.689085+0000","last_change":"2024-05-09T07:13:45.493305+0000","last_active":"2024-05-09T07:39:37.689085+0000","last_peered":"2024-05-09T07:39:37.689085+0000","last_clean":"2024-05-09T07:39:37.689085+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:39:37.689085+0000","last_undegraded":"2024-05-09T07:39:37.689085+0000","last_fullsized":"2024-05-09T07:39:37.689085+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6996","reported_seq":7642,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.270434+0000","last_change":"2024-05-09T07:13:44.469587+0000","last_active":"2024-05-09T07:39:37.270434+0000","last_peered":"2024-05-09T07:39:37.270434+0000","last_clean":"2024-05-09T07:39:37.270434+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:39:37.270434+0000","last_undegraded":"2024-05-09T07:39:37.270434+0000","last_fullsized":"2024-05-09T07:39:37.270434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6996,"log_dups_size":0,"ondisk_log_size":6996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":608,"num_read_kb":4487,"num_write":9817,"num_write_kb":61142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9154","reported_seq":9337,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.629027+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:39:34.629027+0000","last_peered":"2024-05-09T07:39:34.629027+0000","last_clean":"2024-05-09T07:39:34.629027+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:39:34.629027+0000","last_undegraded":"2024-05-09T07:39:34.629027+0000","last_fullsized":"2024-05-09T07:39:34.629027+0000","mapping_epoch":22,"log_start":"27'2834","ondisk_log_start":"27'2834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:00:30.990574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":38100,"num_write":6430,"num_write_kb":884596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2640","reported_seq":2678,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.633575+0000","last_change":"2024-05-09T07:13:45.479039+0000","last_active":"2024-05-09T07:39:37.633575+0000","last_peered":"2024-05-09T07:39:37.633575+0000","last_clean":"2024-05-09T07:39:37.633575+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:39:37.633575+0000","last_undegraded":"2024-05-09T07:39:37.633575+0000","last_fullsized":"2024-05-09T07:39:37.633575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6524","reported_seq":6563,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.270195+0000","last_change":"2024-05-09T07:13:44.475268+0000","last_active":"2024-05-09T07:39:37.270195+0000","last_peered":"2024-05-09T07:39:37.270195+0000","last_clean":"2024-05-09T07:39:37.270195+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:39:37.270195+0000","last_undegraded":"2024-05-09T07:39:37.270195+0000","last_fullsized":"2024-05-09T07:39:37.270195+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6524,"log_dups_size":0,"ondisk_log_size":6524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:36.177031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6527,"num_write_kb":53434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9270","reported_seq":10156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.629480+0000","last_change":"2024-05-09T07:13:49.939668+0000","last_active":"2024-05-09T07:39:34.629480+0000","last_peered":"2024-05-09T07:39:34.629480+0000","last_clean":"2024-05-09T07:39:34.629480+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:39:34.629480+0000","last_undegraded":"2024-05-09T07:39:34.629480+0000","last_fullsized":"2024-05-09T07:39:34.629480+0000","mapping_epoch":22,"log_start":"27'1856","ondisk_log_start":"27'1856","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":853,"num_read_kb":216662,"num_write":6474,"num_write_kb":849337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2527","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.674217+0000","last_change":"2024-05-09T07:14:26.645955+0000","last_active":"2024-05-09T07:39:37.674217+0000","last_peered":"2024-05-09T07:39:37.674217+0000","last_clean":"2024-05-09T07:39:37.674217+0000","last_became_active":"2024-05-09T07:14:26.645530+0000","last_became_peered":"2024-05-09T07:14:26.645530+0000","last_unstale":"2024-05-09T07:39:37.674217+0000","last_undegraded":"2024-05-09T07:39:37.674217+0000","last_fullsized":"2024-05-09T07:39:37.674217+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'8375","reported_seq":8415,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.254135+0000","last_change":"2024-05-09T07:13:44.469711+0000","last_active":"2024-05-09T07:39:37.254135+0000","last_peered":"2024-05-09T07:39:37.254135+0000","last_clean":"2024-05-09T07:39:37.254135+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:39:37.254135+0000","last_undegraded":"2024-05-09T07:39:37.254135+0000","last_fullsized":"2024-05-09T07:39:37.254135+0000","mapping_epoch":17,"log_start":"27'1000","ondisk_log_start":"27'1000","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7375,"log_dups_size":0,"ondisk_log_size":7375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8370,"num_write_kb":75026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9858","reported_seq":10329,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.561712+0000","last_change":"2024-05-09T07:13:49.507640+0000","last_active":"2024-05-09T07:39:34.561712+0000","last_peered":"2024-05-09T07:39:34.561712+0000","last_clean":"2024-05-09T07:39:34.561712+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:39:34.561712+0000","last_undegraded":"2024-05-09T07:39:34.561712+0000","last_fullsized":"2024-05-09T07:39:34.561712+0000","mapping_epoch":22,"log_start":"27'2476","ondisk_log_start":"27'2476","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:53.638392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":438,"num_read_kb":111252,"num_write":7195,"num_write_kb":963969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2574","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.681916+0000","last_change":"2024-05-09T07:13:45.471996+0000","last_active":"2024-05-09T07:39:37.681916+0000","last_peered":"2024-05-09T07:39:37.681916+0000","last_clean":"2024-05-09T07:39:37.681916+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:39:37.681916+0000","last_undegraded":"2024-05-09T07:39:37.681916+0000","last_fullsized":"2024-05-09T07:39:37.681916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2574,"log_dups_size":0,"ondisk_log_size":2574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'9199","reported_seq":9239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.273930+0000","last_change":"2024-05-09T07:13:44.468106+0000","last_active":"2024-05-09T07:39:37.273930+0000","last_peered":"2024-05-09T07:39:37.273930+0000","last_clean":"2024-05-09T07:39:37.273930+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:39:37.273930+0000","last_undegraded":"2024-05-09T07:39:37.273930+0000","last_fullsized":"2024-05-09T07:39:37.273930+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7399,"log_dups_size":0,"ondisk_log_size":7399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9200,"num_write_kb":63563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9948","reported_seq":10113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.559534+0000","last_change":"2024-05-09T07:13:49.502338+0000","last_active":"2024-05-09T07:39:37.559534+0000","last_peered":"2024-05-09T07:39:37.559534+0000","last_clean":"2024-05-09T07:39:37.559534+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:39:37.559534+0000","last_undegraded":"2024-05-09T07:39:37.559534+0000","last_fullsized":"2024-05-09T07:39:37.559534+0000","mapping_epoch":22,"log_start":"27'2616","ondisk_log_start":"27'2616","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:57:17.355783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":33528,"num_write":7177,"num_write_kb":934133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2581","reported_seq":2619,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.627230+0000","last_change":"2024-05-09T07:13:45.479107+0000","last_active":"2024-05-09T07:39:37.627230+0000","last_peered":"2024-05-09T07:39:37.627230+0000","last_clean":"2024-05-09T07:39:37.627230+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:39:37.627230+0000","last_undegraded":"2024-05-09T07:39:37.627230+0000","last_fullsized":"2024-05-09T07:39:37.627230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2581,"log_dups_size":0,"ondisk_log_size":2581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7804","reported_seq":8743,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.572337+0000","last_change":"2024-05-09T07:13:44.471911+0000","last_active":"2024-05-09T07:39:37.572337+0000","last_peered":"2024-05-09T07:39:37.572337+0000","last_clean":"2024-05-09T07:39:37.572337+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:39:37.572337+0000","last_undegraded":"2024-05-09T07:39:37.572337+0000","last_fullsized":"2024-05-09T07:39:37.572337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7804,"log_dups_size":0,"ondisk_log_size":7804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8600773,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":900,"num_read_kb":4675,"num_write":7789,"num_write_kb":62656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9564","reported_seq":9939,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.665427+0000","last_change":"2024-05-09T07:13:49.503130+0000","last_active":"2024-05-09T07:39:34.665427+0000","last_peered":"2024-05-09T07:39:34.665427+0000","last_clean":"2024-05-09T07:39:34.665427+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:39:34.665427+0000","last_undegraded":"2024-05-09T07:39:34.665427+0000","last_fullsized":"2024-05-09T07:39:34.665427+0000","mapping_epoch":22,"log_start":"27'2181","ondisk_log_start":"27'2181","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":87122,"num_write":6858,"num_write_kb":899446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2524","reported_seq":2561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.653121+0000","last_change":"2024-05-09T07:13:45.473751+0000","last_active":"2024-05-09T07:39:37.653121+0000","last_peered":"2024-05-09T07:39:37.653121+0000","last_clean":"2024-05-09T07:39:37.653121+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:39:37.653121+0000","last_undegraded":"2024-05-09T07:39:37.653121+0000","last_fullsized":"2024-05-09T07:39:37.653121+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7852","reported_seq":7891,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.268668+0000","last_change":"2024-05-09T07:13:44.475958+0000","last_active":"2024-05-09T07:39:37.268668+0000","last_peered":"2024-05-09T07:39:37.268668+0000","last_clean":"2024-05-09T07:39:37.268668+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:39:37.268668+0000","last_undegraded":"2024-05-09T07:39:37.268668+0000","last_fullsized":"2024-05-09T07:39:37.268668+0000","mapping_epoch":17,"log_start":"27'800","ondisk_log_start":"27'800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7052,"log_dups_size":0,"ondisk_log_size":7052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:10:20.013293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7854,"num_write_kb":58481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9413","reported_seq":9667,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.561292+0000","last_change":"2024-05-09T07:13:49.939068+0000","last_active":"2024-05-09T07:39:37.561292+0000","last_peered":"2024-05-09T07:39:37.561292+0000","last_clean":"2024-05-09T07:39:37.561292+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:39:37.561292+0000","last_undegraded":"2024-05-09T07:39:37.561292+0000","last_fullsized":"2024-05-09T07:39:37.561292+0000","mapping_epoch":22,"log_start":"27'2042","ondisk_log_start":"27'2042","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":222,"num_read_kb":56388,"num_write":6719,"num_write_kb":1113426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2642","reported_seq":2680,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.710259+0000","last_change":"2024-05-09T07:13:45.489550+0000","last_active":"2024-05-09T07:39:37.710259+0000","last_peered":"2024-05-09T07:39:37.710259+0000","last_clean":"2024-05-09T07:39:37.710259+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:39:37.710259+0000","last_undegraded":"2024-05-09T07:39:37.710259+0000","last_fullsized":"2024-05-09T07:39:37.710259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2642,"log_dups_size":0,"ondisk_log_size":2642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4651","reported_seq":4691,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.248056+0000","last_change":"2024-05-09T07:13:44.467905+0000","last_active":"2024-05-09T07:39:37.248056+0000","last_peered":"2024-05-09T07:39:37.248056+0000","last_clean":"2024-05-09T07:39:37.248056+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:39:37.248056+0000","last_undegraded":"2024-05-09T07:39:37.248056+0000","last_fullsized":"2024-05-09T07:39:37.248056+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4651,"log_dups_size":0,"ondisk_log_size":4651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4655,"num_write_kb":39916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":196,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:37:40.368317+0000","last_change":"2024-05-09T07:13:34.338148+0000","last_active":"2024-05-09T07:37:40.368317+0000","last_peered":"2024-05-09T07:37:40.368317+0000","last_clean":"2024-05-09T07:37:40.368317+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:37:40.368317+0000","last_undegraded":"2024-05-09T07:37:40.368317+0000","last_fullsized":"2024-05-09T07:37:40.368317+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:33.304098+0000","last_clean_scrub_stamp":"2024-05-09T07:13:33.304098+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:30:51.499346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8970","reported_seq":9590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.267024+0000","last_change":"2024-05-09T07:13:49.941229+0000","last_active":"2024-05-09T07:39:34.267024+0000","last_peered":"2024-05-09T07:39:34.267024+0000","last_clean":"2024-05-09T07:39:34.267024+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:39:34.267024+0000","last_undegraded":"2024-05-09T07:39:34.267024+0000","last_fullsized":"2024-05-09T07:39:34.267024+0000","mapping_epoch":22,"log_start":"27'1027","ondisk_log_start":"27'1027","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:41.291683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":587,"num_read_kb":149098,"num_write":6333,"num_write_kb":900398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2565","reported_seq":2586,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.688535+0000","last_change":"2024-05-09T07:14:26.591809+0000","last_active":"2024-05-09T07:39:37.688535+0000","last_peered":"2024-05-09T07:39:37.688535+0000","last_clean":"2024-05-09T07:39:37.688535+0000","last_became_active":"2024-05-09T07:14:26.591447+0000","last_became_peered":"2024-05-09T07:14:26.591447+0000","last_unstale":"2024-05-09T07:39:37.688535+0000","last_undegraded":"2024-05-09T07:39:37.688535+0000","last_fullsized":"2024-05-09T07:39:37.688535+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"27'7875","reported_seq":7913,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.270069+0000","last_change":"2024-05-09T07:13:44.472204+0000","last_active":"2024-05-09T07:39:37.270069+0000","last_peered":"2024-05-09T07:39:37.270069+0000","last_clean":"2024-05-09T07:39:37.270069+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:39:37.270069+0000","last_undegraded":"2024-05-09T07:39:37.270069+0000","last_fullsized":"2024-05-09T07:39:37.270069+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7875,"log_dups_size":0,"ondisk_log_size":7875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7879,"num_write_kb":58494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9733","reported_seq":10214,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.461178+0000","last_change":"2024-05-09T07:13:49.939353+0000","last_active":"2024-05-09T07:39:34.461178+0000","last_peered":"2024-05-09T07:39:34.461178+0000","last_clean":"2024-05-09T07:39:34.461178+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:39:34.461178+0000","last_undegraded":"2024-05-09T07:39:34.461178+0000","last_fullsized":"2024-05-09T07:39:34.461178+0000","mapping_epoch":22,"log_start":"27'1834","ondisk_log_start":"27'1834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7899,"log_dups_size":0,"ondisk_log_size":7899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":449,"num_read_kb":114046,"num_write":7019,"num_write_kb":962272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2589","reported_seq":2626,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.710502+0000","last_change":"2024-05-09T07:13:45.483265+0000","last_active":"2024-05-09T07:39:37.710502+0000","last_peered":"2024-05-09T07:39:37.710502+0000","last_clean":"2024-05-09T07:39:37.710502+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:39:37.710502+0000","last_undegraded":"2024-05-09T07:39:37.710502+0000","last_fullsized":"2024-05-09T07:39:37.710502+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7545","reported_seq":7584,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.238576+0000","last_change":"2024-05-09T07:13:44.470447+0000","last_active":"2024-05-09T07:39:37.238576+0000","last_peered":"2024-05-09T07:39:37.238576+0000","last_clean":"2024-05-09T07:39:37.238576+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:39:37.238576+0000","last_undegraded":"2024-05-09T07:39:37.238576+0000","last_fullsized":"2024-05-09T07:39:37.238576+0000","mapping_epoch":17,"log_start":"27'200","ondisk_log_start":"27'200","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7345,"log_dups_size":0,"ondisk_log_size":7345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:14.217140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7546,"num_write_kb":58281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9829","reported_seq":10112,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.660406+0000","last_change":"2024-05-09T07:13:49.506750+0000","last_active":"2024-05-09T07:39:34.660406+0000","last_peered":"2024-05-09T07:39:34.660406+0000","last_clean":"2024-05-09T07:39:34.660406+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:39:34.660406+0000","last_undegraded":"2024-05-09T07:39:34.660406+0000","last_fullsized":"2024-05-09T07:39:34.660406+0000","mapping_epoch":22,"log_start":"27'2499","ondisk_log_start":"27'2499","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":251,"num_read_kb":63754,"num_write":7056,"num_write_kb":893799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2456","reported_seq":2492,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.693374+0000","last_change":"2024-05-09T07:13:45.497993+0000","last_active":"2024-05-09T07:39:37.693374+0000","last_peered":"2024-05-09T07:39:37.693374+0000","last_clean":"2024-05-09T07:39:37.693374+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:39:37.693374+0000","last_undegraded":"2024-05-09T07:39:37.693374+0000","last_fullsized":"2024-05-09T07:39:37.693374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2456,"log_dups_size":0,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:48.910693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8863","reported_seq":8901,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.230247+0000","last_change":"2024-05-09T07:13:44.471561+0000","last_active":"2024-05-09T07:39:37.230247+0000","last_peered":"2024-05-09T07:39:37.230247+0000","last_clean":"2024-05-09T07:39:37.230247+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:39:37.230247+0000","last_undegraded":"2024-05-09T07:39:37.230247+0000","last_fullsized":"2024-05-09T07:39:37.230247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":8863,"log_dups_size":0,"ondisk_log_size":8863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8854,"num_write_kb":79483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9365","reported_seq":10380,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.626710+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:39:34.626710+0000","last_peered":"2024-05-09T07:39:34.626710+0000","last_clean":"2024-05-09T07:39:34.626710+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:39:34.626710+0000","last_undegraded":"2024-05-09T07:39:34.626710+0000","last_fullsized":"2024-05-09T07:39:34.626710+0000","mapping_epoch":22,"log_start":"27'1982","ondisk_log_start":"27'1982","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:53:23.633010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":983,"num_read_kb":249682,"num_write":6568,"num_write_kb":1037240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2557","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.667273+0000","last_change":"2024-05-09T07:13:45.488596+0000","last_active":"2024-05-09T07:39:37.667273+0000","last_peered":"2024-05-09T07:39:37.667273+0000","last_clean":"2024-05-09T07:39:37.667273+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:39:37.667273+0000","last_undegraded":"2024-05-09T07:39:37.667273+0000","last_fullsized":"2024-05-09T07:39:37.667273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:24:30.454293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5465","reported_seq":5503,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:17.759415+0000","last_change":"2024-05-09T07:13:44.470052+0000","last_active":"2024-05-09T07:39:17.759415+0000","last_peered":"2024-05-09T07:39:17.759415+0000","last_clean":"2024-05-09T07:39:17.759415+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:39:17.759415+0000","last_undegraded":"2024-05-09T07:39:17.759415+0000","last_fullsized":"2024-05-09T07:39:17.759415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5465,"log_dups_size":0,"ondisk_log_size":5465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:44:08.638382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5458,"num_write_kb":44489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9305","reported_seq":9864,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.591193+0000","last_change":"2024-05-09T07:13:49.506082+0000","last_active":"2024-05-09T07:39:34.591193+0000","last_peered":"2024-05-09T07:39:34.591193+0000","last_clean":"2024-05-09T07:39:34.591193+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:39:34.591193+0000","last_undegraded":"2024-05-09T07:39:34.591193+0000","last_fullsized":"2024-05-09T07:39:34.591193+0000","mapping_epoch":22,"log_start":"27'2103","ondisk_log_start":"27'2103","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:56.094406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":527,"num_read_kb":133858,"num_write":6613,"num_write_kb":868619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2624","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.665457+0000","last_change":"2024-05-09T07:13:45.497815+0000","last_active":"2024-05-09T07:39:37.665457+0000","last_peered":"2024-05-09T07:39:37.665457+0000","last_clean":"2024-05-09T07:39:37.665457+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:39:37.665457+0000","last_undegraded":"2024-05-09T07:39:37.665457+0000","last_fullsized":"2024-05-09T07:39:37.665457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3468","reported_seq":3506,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.203831+0000","last_change":"2024-05-09T07:13:44.468622+0000","last_active":"2024-05-09T07:39:37.203831+0000","last_peered":"2024-05-09T07:39:37.203831+0000","last_clean":"2024-05-09T07:39:37.203831+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:39:37.203831+0000","last_undegraded":"2024-05-09T07:39:37.203831+0000","last_fullsized":"2024-05-09T07:39:37.203831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":3468,"log_dups_size":0,"ondisk_log_size":3468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3482,"num_write_kb":39113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9477","reported_seq":10052,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.565062+0000","last_change":"2024-05-09T07:13:49.506194+0000","last_active":"2024-05-09T07:39:34.565062+0000","last_peered":"2024-05-09T07:39:34.565062+0000","last_clean":"2024-05-09T07:39:34.565062+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:39:34.565062+0000","last_undegraded":"2024-05-09T07:39:34.565062+0000","last_fullsized":"2024-05-09T07:39:34.565062+0000","mapping_epoch":22,"log_start":"27'2114","ondisk_log_start":"27'2114","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:55.875377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":138176,"num_write":6755,"num_write_kb":1181456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2498","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.709516+0000","last_change":"2024-05-09T07:13:45.484113+0000","last_active":"2024-05-09T07:39:37.709516+0000","last_peered":"2024-05-09T07:39:37.709516+0000","last_clean":"2024-05-09T07:39:37.709516+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:39:37.709516+0000","last_undegraded":"2024-05-09T07:39:37.709516+0000","last_fullsized":"2024-05-09T07:39:37.709516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4968","reported_seq":5007,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.276315+0000","last_change":"2024-05-09T07:13:44.465639+0000","last_active":"2024-05-09T07:39:37.276315+0000","last_peered":"2024-05-09T07:39:37.276315+0000","last_clean":"2024-05-09T07:39:37.276315+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:39:37.276315+0000","last_undegraded":"2024-05-09T07:39:37.276315+0000","last_fullsized":"2024-05-09T07:39:37.276315+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":4979,"num_write_kb":44143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9318","reported_seq":9621,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.662016+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:39:34.662016+0000","last_peered":"2024-05-09T07:39:34.662016+0000","last_clean":"2024-05-09T07:39:34.662016+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:39:34.662016+0000","last_undegraded":"2024-05-09T07:39:34.662016+0000","last_fullsized":"2024-05-09T07:39:34.662016+0000","mapping_epoch":22,"log_start":"27'3038","ondisk_log_start":"27'3038","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:34:57.562656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":69088,"num_write":6517,"num_write_kb":924175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2603","reported_seq":2639,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.702035+0000","last_change":"2024-05-09T07:13:45.498431+0000","last_active":"2024-05-09T07:39:37.702035+0000","last_peered":"2024-05-09T07:39:37.702035+0000","last_clean":"2024-05-09T07:39:37.702035+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:39:37.702035+0000","last_undegraded":"2024-05-09T07:39:37.702035+0000","last_fullsized":"2024-05-09T07:39:37.702035+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:21:49.607969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2651","reported_seq":2690,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.209674+0000","last_change":"2024-05-09T07:13:44.460641+0000","last_active":"2024-05-09T07:39:37.209674+0000","last_peered":"2024-05-09T07:39:37.209674+0000","last_clean":"2024-05-09T07:39:37.209674+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:39:37.209674+0000","last_undegraded":"2024-05-09T07:39:37.209674+0000","last_fullsized":"2024-05-09T07:39:37.209674+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":2651,"log_dups_size":0,"ondisk_log_size":2651,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:30:57.025732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2652,"num_write_kb":30266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9224","reported_seq":9452,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.629589+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:39:34.629589+0000","last_peered":"2024-05-09T07:39:34.629589+0000","last_clean":"2024-05-09T07:39:34.629589+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:39:34.629589+0000","last_undegraded":"2024-05-09T07:39:34.629589+0000","last_fullsized":"2024-05-09T07:39:34.629589+0000","mapping_epoch":22,"log_start":"27'2939","ondisk_log_start":"27'2939","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6285,"log_dups_size":0,"ondisk_log_size":6285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:57:54.556570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":197,"num_read_kb":50038,"num_write":6664,"num_write_kb":910804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2562","reported_seq":2598,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.674386+0000","last_change":"2024-05-09T07:13:45.492779+0000","last_active":"2024-05-09T07:39:37.674386+0000","last_peered":"2024-05-09T07:39:37.674386+0000","last_clean":"2024-05-09T07:39:37.674386+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:39:37.674386+0000","last_undegraded":"2024-05-09T07:39:37.674386+0000","last_fullsized":"2024-05-09T07:39:37.674386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6997","reported_seq":7035,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.229011+0000","last_change":"2024-05-09T07:13:44.466374+0000","last_active":"2024-05-09T07:39:37.229011+0000","last_peered":"2024-05-09T07:39:37.229011+0000","last_clean":"2024-05-09T07:39:37.229011+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:39:37.229011+0000","last_undegraded":"2024-05-09T07:39:37.229011+0000","last_fullsized":"2024-05-09T07:39:37.229011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6997,"log_dups_size":0,"ondisk_log_size":6997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6989,"num_write_kb":57820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9732","reported_seq":10000,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.649697+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:39:34.649697+0000","last_peered":"2024-05-09T07:39:34.649697+0000","last_clean":"2024-05-09T07:39:34.649697+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:39:34.649697+0000","last_undegraded":"2024-05-09T07:39:34.649697+0000","last_fullsized":"2024-05-09T07:39:34.649697+0000","mapping_epoch":22,"log_start":"27'2359","ondisk_log_start":"27'2359","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7373,"log_dups_size":0,"ondisk_log_size":7373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:03:54.421786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":60198,"num_write":7045,"num_write_kb":1050276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2506","reported_seq":2541,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.702610+0000","last_change":"2024-05-09T07:13:45.486906+0000","last_active":"2024-05-09T07:39:37.702610+0000","last_peered":"2024-05-09T07:39:37.702610+0000","last_clean":"2024-05-09T07:39:37.702610+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:39:37.702610+0000","last_undegraded":"2024-05-09T07:39:37.702610+0000","last_fullsized":"2024-05-09T07:39:37.702610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2506,"log_dups_size":0,"ondisk_log_size":2506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10543","reported_seq":11209,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.253648+0000","last_change":"2024-05-09T07:13:44.466450+0000","last_active":"2024-05-09T07:39:37.253648+0000","last_peered":"2024-05-09T07:39:37.253648+0000","last_clean":"2024-05-09T07:39:37.253648+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:39:37.253648+0000","last_undegraded":"2024-05-09T07:39:37.253648+0000","last_fullsized":"2024-05-09T07:39:37.253648+0000","mapping_epoch":17,"log_start":"27'2600","ondisk_log_start":"27'2600","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":627,"num_read_kb":4520,"num_write":10536,"num_write_kb":76795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9196","reported_seq":9439,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.665947+0000","last_change":"2024-05-09T07:14:26.592176+0000","last_active":"2024-05-09T07:39:34.665947+0000","last_peered":"2024-05-09T07:39:34.665947+0000","last_clean":"2024-05-09T07:39:34.665947+0000","last_became_active":"2024-05-09T07:14:26.590721+0000","last_became_peered":"2024-05-09T07:14:26.590721+0000","last_unstale":"2024-05-09T07:39:34.665947+0000","last_undegraded":"2024-05-09T07:39:34.665947+0000","last_fullsized":"2024-05-09T07:39:34.665947+0000","mapping_epoch":26,"log_start":"27'1828","ondisk_log_start":"27'1828","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:30:38.451332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":56134,"num_write":6509,"num_write_kb":978701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"27'2577","reported_seq":2612,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.660049+0000","last_change":"2024-05-09T07:13:45.494302+0000","last_active":"2024-05-09T07:39:37.660049+0000","last_peered":"2024-05-09T07:39:37.660049+0000","last_clean":"2024-05-09T07:39:37.660049+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:39:37.660049+0000","last_undegraded":"2024-05-09T07:39:37.660049+0000","last_fullsized":"2024-05-09T07:39:37.660049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:46:15.220812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6179","reported_seq":6217,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.279928+0000","last_change":"2024-05-09T07:13:44.471190+0000","last_active":"2024-05-09T07:39:37.279928+0000","last_peered":"2024-05-09T07:39:37.279928+0000","last_clean":"2024-05-09T07:39:37.279928+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:39:37.279928+0000","last_undegraded":"2024-05-09T07:39:37.279928+0000","last_fullsized":"2024-05-09T07:39:37.279928+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6179,"log_dups_size":0,"ondisk_log_size":6179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6185,"num_write_kb":41396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9171","reported_seq":9514,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.425848+0000","last_change":"2024-05-09T07:14:26.596397+0000","last_active":"2024-05-09T07:39:34.425848+0000","last_peered":"2024-05-09T07:39:34.425848+0000","last_clean":"2024-05-09T07:39:34.425848+0000","last_became_active":"2024-05-09T07:14:26.595936+0000","last_became_peered":"2024-05-09T07:14:26.595936+0000","last_unstale":"2024-05-09T07:39:34.425848+0000","last_undegraded":"2024-05-09T07:39:34.425848+0000","last_fullsized":"2024-05-09T07:39:34.425848+0000","mapping_epoch":26,"log_start":"27'2010","ondisk_log_start":"27'2010","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7161,"log_dups_size":0,"ondisk_log_size":7161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:45.938473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":305,"num_read_kb":77470,"num_write":6446,"num_write_kb":989255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2611","reported_seq":2646,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.721690+0000","last_change":"2024-05-09T07:13:45.489170+0000","last_active":"2024-05-09T07:39:37.721690+0000","last_peered":"2024-05-09T07:39:37.721690+0000","last_clean":"2024-05-09T07:39:37.721690+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:39:37.721690+0000","last_undegraded":"2024-05-09T07:39:37.721690+0000","last_fullsized":"2024-05-09T07:39:37.721690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5741","reported_seq":5779,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.268106+0000","last_change":"2024-05-09T07:13:44.470512+0000","last_active":"2024-05-09T07:39:37.268106+0000","last_peered":"2024-05-09T07:39:37.268106+0000","last_clean":"2024-05-09T07:39:37.268106+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:39:37.268106+0000","last_undegraded":"2024-05-09T07:39:37.268106+0000","last_fullsized":"2024-05-09T07:39:37.268106+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5741,"log_dups_size":0,"ondisk_log_size":5741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:45.681470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5742,"num_write_kb":52939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9121","reported_seq":9488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:38.186322+0000","last_change":"2024-05-09T07:13:49.504191+0000","last_active":"2024-05-09T07:39:38.186322+0000","last_peered":"2024-05-09T07:39:38.186322+0000","last_clean":"2024-05-09T07:39:38.186322+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:39:38.186322+0000","last_undegraded":"2024-05-09T07:39:38.186322+0000","last_fullsized":"2024-05-09T07:39:38.186322+0000","mapping_epoch":22,"log_start":"27'2805","ondisk_log_start":"27'2805","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:10:04.142263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":335,"num_read_kb":85090,"num_write":6449,"num_write_kb":967440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2587","reported_seq":2623,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.688354+0000","last_change":"2024-05-09T07:13:45.492759+0000","last_active":"2024-05-09T07:39:37.688354+0000","last_peered":"2024-05-09T07:39:37.688354+0000","last_clean":"2024-05-09T07:39:37.688354+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:39:37.688354+0000","last_undegraded":"2024-05-09T07:39:37.688354+0000","last_fullsized":"2024-05-09T07:39:37.688354+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:12:59.014675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'5946","reported_seq":6850,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.272006+0000","last_change":"2024-05-09T07:13:44.471358+0000","last_active":"2024-05-09T07:39:37.272006+0000","last_peered":"2024-05-09T07:39:37.272006+0000","last_clean":"2024-05-09T07:39:37.272006+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:39:37.272006+0000","last_undegraded":"2024-05-09T07:39:37.272006+0000","last_fullsized":"2024-05-09T07:39:37.272006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5946,"log_dups_size":0,"ondisk_log_size":5946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:49.721712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":867,"num_read_kb":4659,"num_write":5947,"num_write_kb":49048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9603","reported_seq":9991,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.584782+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:39:34.584782+0000","last_peered":"2024-05-09T07:39:34.584782+0000","last_clean":"2024-05-09T07:39:34.584782+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:39:34.584782+0000","last_undegraded":"2024-05-09T07:39:34.584782+0000","last_fullsized":"2024-05-09T07:39:34.584782+0000","mapping_epoch":22,"log_start":"27'3323","ondisk_log_start":"27'3323","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:50:48.896459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6822,"num_write_kb":1059940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9164","reported_seq":9391,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.560748+0000","last_change":"2024-05-09T07:13:49.940621+0000","last_active":"2024-05-09T07:39:37.560748+0000","last_peered":"2024-05-09T07:39:37.560748+0000","last_clean":"2024-05-09T07:39:37.560748+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:39:37.560748+0000","last_undegraded":"2024-05-09T07:39:37.560748+0000","last_fullsized":"2024-05-09T07:39:37.560748+0000","mapping_epoch":22,"log_start":"27'1813","ondisk_log_start":"27'1813","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7351,"log_dups_size":0,"ondisk_log_size":7351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:31:23.649152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":49276,"num_write":6561,"num_write_kb":912358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2535","reported_seq":2571,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.684140+0000","last_change":"2024-05-09T07:13:45.477698+0000","last_active":"2024-05-09T07:39:37.684140+0000","last_peered":"2024-05-09T07:39:37.684140+0000","last_clean":"2024-05-09T07:39:37.684140+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:39:37.684140+0000","last_undegraded":"2024-05-09T07:39:37.684140+0000","last_fullsized":"2024-05-09T07:39:37.684140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2535,"log_dups_size":0,"ondisk_log_size":2535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9071","reported_seq":9109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.268802+0000","last_change":"2024-05-09T07:13:44.466479+0000","last_active":"2024-05-09T07:39:37.268802+0000","last_peered":"2024-05-09T07:39:37.268802+0000","last_clean":"2024-05-09T07:39:37.268802+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:39:37.268802+0000","last_undegraded":"2024-05-09T07:39:37.268802+0000","last_fullsized":"2024-05-09T07:39:37.268802+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9068,"num_write_kb":67566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8844","reported_seq":10030,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.572579+0000","last_change":"2024-05-09T07:13:49.507227+0000","last_active":"2024-05-09T07:39:34.572579+0000","last_peered":"2024-05-09T07:39:34.572579+0000","last_clean":"2024-05-09T07:39:34.572579+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:39:34.572579+0000","last_undegraded":"2024-05-09T07:39:34.572579+0000","last_fullsized":"2024-05-09T07:39:34.572579+0000","mapping_epoch":22,"log_start":"27'900","ondisk_log_start":"27'900","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7944,"log_dups_size":0,"ondisk_log_size":7944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:52.941116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1153,"num_read_kb":292862,"num_write":6082,"num_write_kb":877058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2633","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.709037+0000","last_change":"2024-05-09T07:13:45.478917+0000","last_active":"2024-05-09T07:39:37.709037+0000","last_peered":"2024-05-09T07:39:37.709037+0000","last_clean":"2024-05-09T07:39:37.709037+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:39:37.709037+0000","last_undegraded":"2024-05-09T07:39:37.709037+0000","last_fullsized":"2024-05-09T07:39:37.709037+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2633,"log_dups_size":0,"ondisk_log_size":2633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:59.693963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5356","reported_seq":5394,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.250321+0000","last_change":"2024-05-09T07:13:44.469836+0000","last_active":"2024-05-09T07:39:37.250321+0000","last_peered":"2024-05-09T07:39:37.250321+0000","last_clean":"2024-05-09T07:39:37.250321+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:39:37.250321+0000","last_undegraded":"2024-05-09T07:39:37.250321+0000","last_fullsized":"2024-05-09T07:39:37.250321+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":5356,"log_dups_size":0,"ondisk_log_size":5356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:15:55.791194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5356,"num_write_kb":44481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9697","reported_seq":9735,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.263246+0000","last_change":"2024-05-09T07:13:44.464626+0000","last_active":"2024-05-09T07:39:37.263246+0000","last_peered":"2024-05-09T07:39:37.263246+0000","last_clean":"2024-05-09T07:39:37.263246+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:39:37.263246+0000","last_undegraded":"2024-05-09T07:39:37.263246+0000","last_fullsized":"2024-05-09T07:39:37.263246+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":7897,"log_dups_size":0,"ondisk_log_size":7897,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:48:07.859152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9694,"num_write_kb":72019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2561","reported_seq":2596,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.711419+0000","last_change":"2024-05-09T07:13:45.473893+0000","last_active":"2024-05-09T07:39:37.711419+0000","last_peered":"2024-05-09T07:39:37.711419+0000","last_clean":"2024-05-09T07:39:37.711419+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:39:37.711419+0000","last_undegraded":"2024-05-09T07:39:37.711419+0000","last_fullsized":"2024-05-09T07:39:37.711419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9401","reported_seq":9456,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.576170+0000","last_change":"2024-05-09T07:13:49.494828+0000","last_active":"2024-05-09T07:39:34.576170+0000","last_peered":"2024-05-09T07:39:34.576170+0000","last_clean":"2024-05-09T07:39:34.576170+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:39:34.576170+0000","last_undegraded":"2024-05-09T07:39:34.576170+0000","last_fullsized":"2024-05-09T07:39:34.576170+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:12.044577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":6096,"num_write":6603,"num_write_kb":983207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8029","reported_seq":8067,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.266280+0000","last_change":"2024-05-09T07:13:44.465848+0000","last_active":"2024-05-09T07:39:37.266280+0000","last_peered":"2024-05-09T07:39:37.266280+0000","last_clean":"2024-05-09T07:39:37.266280+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:39:37.266280+0000","last_undegraded":"2024-05-09T07:39:37.266280+0000","last_fullsized":"2024-05-09T07:39:37.266280+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:43.444485+0000","last_clean_scrub_stamp":"2024-05-09T07:13:43.444485+0000","objects_scrubbed":0,"log_size":6329,"log_dups_size":0,"ondisk_log_size":6329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:42:57.417986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8031,"num_write_kb":66635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2603","reported_seq":2638,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.626428+0000","last_change":"2024-05-09T07:13:45.476599+0000","last_active":"2024-05-09T07:39:37.626428+0000","last_peered":"2024-05-09T07:39:37.626428+0000","last_clean":"2024-05-09T07:39:37.626428+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:39:37.626428+0000","last_undegraded":"2024-05-09T07:39:37.626428+0000","last_fullsized":"2024-05-09T07:39:37.626428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:43:46.685143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9830","reported_seq":9982,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.611050+0000","last_change":"2024-05-09T07:13:49.504616+0000","last_active":"2024-05-09T07:39:34.611050+0000","last_peered":"2024-05-09T07:39:34.611050+0000","last_clean":"2024-05-09T07:39:34.611050+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:39:34.611050+0000","last_undegraded":"2024-05-09T07:39:34.611050+0000","last_fullsized":"2024-05-09T07:39:34.611050+0000","mapping_epoch":22,"log_start":"27'3543","ondisk_log_start":"27'3543","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:41:30.521354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":30734,"num_write":7017,"num_write_kb":1030226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2510","reported_seq":2548,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.709713+0000","last_change":"2024-05-09T07:13:45.489391+0000","last_active":"2024-05-09T07:39:37.709713+0000","last_peered":"2024-05-09T07:39:37.709713+0000","last_clean":"2024-05-09T07:39:37.709713+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:39:37.709713+0000","last_undegraded":"2024-05-09T07:39:37.709713+0000","last_fullsized":"2024-05-09T07:39:37.709713+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2625","reported_seq":2662,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.664991+0000","last_change":"2024-05-09T07:13:45.486665+0000","last_active":"2024-05-09T07:39:37.664991+0000","last_peered":"2024-05-09T07:39:37.664991+0000","last_clean":"2024-05-09T07:39:37.664991+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:39:37.664991+0000","last_undegraded":"2024-05-09T07:39:37.664991+0000","last_fullsized":"2024-05-09T07:39:37.664991+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:56:54.841890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2528","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.716353+0000","last_change":"2024-05-09T07:13:45.490637+0000","last_active":"2024-05-09T07:39:37.716353+0000","last_peered":"2024-05-09T07:39:37.716353+0000","last_clean":"2024-05-09T07:39:37.716353+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:39:37.716353+0000","last_undegraded":"2024-05-09T07:39:37.716353+0000","last_fullsized":"2024-05-09T07:39:37.716353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2612","reported_seq":2649,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.705714+0000","last_change":"2024-05-09T07:13:45.475814+0000","last_active":"2024-05-09T07:39:37.705714+0000","last_peered":"2024-05-09T07:39:37.705714+0000","last_clean":"2024-05-09T07:39:37.705714+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:39:37.705714+0000","last_undegraded":"2024-05-09T07:39:37.705714+0000","last_fullsized":"2024-05-09T07:39:37.705714+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2606","reported_seq":2643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.684917+0000","last_change":"2024-05-09T07:13:45.483470+0000","last_active":"2024-05-09T07:39:37.684917+0000","last_peered":"2024-05-09T07:39:37.684917+0000","last_clean":"2024-05-09T07:39:37.684917+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:39:37.684917+0000","last_undegraded":"2024-05-09T07:39:37.684917+0000","last_fullsized":"2024-05-09T07:39:37.684917+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2485","reported_seq":2522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.644744+0000","last_change":"2024-05-09T07:13:45.475772+0000","last_active":"2024-05-09T07:39:37.644744+0000","last_peered":"2024-05-09T07:39:37.644744+0000","last_clean":"2024-05-09T07:39:37.644744+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:39:37.644744+0000","last_undegraded":"2024-05-09T07:39:37.644744+0000","last_fullsized":"2024-05-09T07:39:37.644744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2622","reported_seq":2659,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.718601+0000","last_change":"2024-05-09T07:13:45.494038+0000","last_active":"2024-05-09T07:39:37.718601+0000","last_peered":"2024-05-09T07:39:37.718601+0000","last_clean":"2024-05-09T07:39:37.718601+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:39:37.718601+0000","last_undegraded":"2024-05-09T07:39:37.718601+0000","last_fullsized":"2024-05-09T07:39:37.718601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2622,"log_dups_size":0,"ondisk_log_size":2622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2594","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.663547+0000","last_change":"2024-05-09T07:13:45.483720+0000","last_active":"2024-05-09T07:39:37.663547+0000","last_peered":"2024-05-09T07:39:37.663547+0000","last_clean":"2024-05-09T07:39:37.663547+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:39:37.663547+0000","last_undegraded":"2024-05-09T07:39:37.663547+0000","last_fullsized":"2024-05-09T07:39:37.663547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2594,"log_dups_size":0,"ondisk_log_size":2594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2576","reported_seq":2597,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.685455+0000","last_change":"2024-05-09T07:14:26.646404+0000","last_active":"2024-05-09T07:39:37.685455+0000","last_peered":"2024-05-09T07:39:37.685455+0000","last_clean":"2024-05-09T07:39:37.685455+0000","last_became_active":"2024-05-09T07:14:26.646058+0000","last_became_peered":"2024-05-09T07:14:26.646058+0000","last_unstale":"2024-05-09T07:39:37.685455+0000","last_undegraded":"2024-05-09T07:39:37.685455+0000","last_fullsized":"2024-05-09T07:39:37.685455+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'2541","reported_seq":2578,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.696800+0000","last_change":"2024-05-09T07:13:45.486581+0000","last_active":"2024-05-09T07:39:37.696800+0000","last_peered":"2024-05-09T07:39:37.696800+0000","last_clean":"2024-05-09T07:39:37.696800+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:39:37.696800+0000","last_undegraded":"2024-05-09T07:39:37.696800+0000","last_fullsized":"2024-05-09T07:39:37.696800+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2609","reported_seq":2646,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.715235+0000","last_change":"2024-05-09T07:13:45.493759+0000","last_active":"2024-05-09T07:39:37.715235+0000","last_peered":"2024-05-09T07:39:37.715235+0000","last_clean":"2024-05-09T07:39:37.715235+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:39:37.715235+0000","last_undegraded":"2024-05-09T07:39:37.715235+0000","last_fullsized":"2024-05-09T07:39:37.715235+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2609,"log_dups_size":0,"ondisk_log_size":2609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:32:35.438195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2621","reported_seq":2657,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.707814+0000","last_change":"2024-05-09T07:13:45.493919+0000","last_active":"2024-05-09T07:39:37.707814+0000","last_peered":"2024-05-09T07:39:37.707814+0000","last_clean":"2024-05-09T07:39:37.707814+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:39:37.707814+0000","last_undegraded":"2024-05-09T07:39:37.707814+0000","last_fullsized":"2024-05-09T07:39:37.707814+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2621,"log_dups_size":0,"ondisk_log_size":2621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2611","reported_seq":2648,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.695378+0000","last_change":"2024-05-09T07:13:45.491655+0000","last_active":"2024-05-09T07:39:37.695378+0000","last_peered":"2024-05-09T07:39:37.695378+0000","last_clean":"2024-05-09T07:39:37.695378+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:39:37.695378+0000","last_undegraded":"2024-05-09T07:39:37.695378+0000","last_fullsized":"2024-05-09T07:39:37.695378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:49:07.488590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2615","reported_seq":2651,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.702442+0000","last_change":"2024-05-09T07:13:45.491233+0000","last_active":"2024-05-09T07:39:37.702442+0000","last_peered":"2024-05-09T07:39:37.702442+0000","last_clean":"2024-05-09T07:39:37.702442+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:39:37.702442+0000","last_undegraded":"2024-05-09T07:39:37.702442+0000","last_fullsized":"2024-05-09T07:39:37.702442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2615,"log_dups_size":0,"ondisk_log_size":2615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.037483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2554","reported_seq":2590,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.658707+0000","last_change":"2024-05-09T07:13:45.488429+0000","last_active":"2024-05-09T07:39:37.658707+0000","last_peered":"2024-05-09T07:39:37.658707+0000","last_clean":"2024-05-09T07:39:37.658707+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:39:37.658707+0000","last_undegraded":"2024-05-09T07:39:37.658707+0000","last_fullsized":"2024-05-09T07:39:37.658707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:53:37.180703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2625","reported_seq":2662,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.731978+0000","last_change":"2024-05-09T07:13:45.496331+0000","last_active":"2024-05-09T07:39:37.731978+0000","last_peered":"2024-05-09T07:39:37.731978+0000","last_clean":"2024-05-09T07:39:37.731978+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:39:37.731978+0000","last_undegraded":"2024-05-09T07:39:37.731978+0000","last_fullsized":"2024-05-09T07:39:37.731978+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:44:52.063982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2558","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.679195+0000","last_change":"2024-05-09T07:13:45.489638+0000","last_active":"2024-05-09T07:39:37.679195+0000","last_peered":"2024-05-09T07:39:37.679195+0000","last_clean":"2024-05-09T07:39:37.679195+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:39:37.679195+0000","last_undegraded":"2024-05-09T07:39:37.679195+0000","last_fullsized":"2024-05-09T07:39:37.679195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:55:40.819428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2678","reported_seq":2715,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.678072+0000","last_change":"2024-05-09T07:13:45.472925+0000","last_active":"2024-05-09T07:39:37.678072+0000","last_peered":"2024-05-09T07:39:37.678072+0000","last_clean":"2024-05-09T07:39:37.678072+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:39:37.678072+0000","last_undegraded":"2024-05-09T07:39:37.678072+0000","last_fullsized":"2024-05-09T07:39:37.678072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2529","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.678113+0000","last_change":"2024-05-09T07:13:45.476809+0000","last_active":"2024-05-09T07:39:37.678113+0000","last_peered":"2024-05-09T07:39:37.678113+0000","last_clean":"2024-05-09T07:39:37.678113+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:39:37.678113+0000","last_undegraded":"2024-05-09T07:39:37.678113+0000","last_fullsized":"2024-05-09T07:39:37.678113+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:08.014154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2521","reported_seq":2557,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.701277+0000","last_change":"2024-05-09T07:13:45.491204+0000","last_active":"2024-05-09T07:39:37.701277+0000","last_peered":"2024-05-09T07:39:37.701277+0000","last_clean":"2024-05-09T07:39:37.701277+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:39:37.701277+0000","last_undegraded":"2024-05-09T07:39:37.701277+0000","last_fullsized":"2024-05-09T07:39:37.701277+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:48:47.046895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2490","reported_seq":2525,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.655011+0000","last_change":"2024-05-09T07:13:45.496499+0000","last_active":"2024-05-09T07:39:37.655011+0000","last_peered":"2024-05-09T07:39:37.655011+0000","last_clean":"2024-05-09T07:39:37.655011+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:39:37.655011+0000","last_undegraded":"2024-05-09T07:39:37.655011+0000","last_fullsized":"2024-05-09T07:39:37.655011+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2490,"log_dups_size":0,"ondisk_log_size":2490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:09:50.725384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2514","reported_seq":2550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.688808+0000","last_change":"2024-05-09T07:13:45.491782+0000","last_active":"2024-05-09T07:39:37.688808+0000","last_peered":"2024-05-09T07:39:37.688808+0000","last_clean":"2024-05-09T07:39:37.688808+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:39:37.688808+0000","last_undegraded":"2024-05-09T07:39:37.688808+0000","last_fullsized":"2024-05-09T07:39:37.688808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:59:22.192120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2640","reported_seq":2676,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.713013+0000","last_change":"2024-05-09T07:13:45.486501+0000","last_active":"2024-05-09T07:39:37.713013+0000","last_peered":"2024-05-09T07:39:37.713013+0000","last_clean":"2024-05-09T07:39:37.713013+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:39:37.713013+0000","last_undegraded":"2024-05-09T07:39:37.713013+0000","last_fullsized":"2024-05-09T07:39:37.713013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2612","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.728318+0000","last_change":"2024-05-09T07:13:45.492797+0000","last_active":"2024-05-09T07:39:37.728318+0000","last_peered":"2024-05-09T07:39:37.728318+0000","last_clean":"2024-05-09T07:39:37.728318+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:39:37.728318+0000","last_undegraded":"2024-05-09T07:39:37.728318+0000","last_fullsized":"2024-05-09T07:39:37.728318+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:55:59.286453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2548","reported_seq":2584,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.714336+0000","last_change":"2024-05-09T07:13:45.489764+0000","last_active":"2024-05-09T07:39:37.714336+0000","last_peered":"2024-05-09T07:39:37.714336+0000","last_clean":"2024-05-09T07:39:37.714336+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:39:37.714336+0000","last_undegraded":"2024-05-09T07:39:37.714336+0000","last_fullsized":"2024-05-09T07:39:37.714336+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:05:14.934568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2567","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.652759+0000","last_change":"2024-05-09T07:13:45.486315+0000","last_active":"2024-05-09T07:39:37.652759+0000","last_peered":"2024-05-09T07:39:37.652759+0000","last_clean":"2024-05-09T07:39:37.652759+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:39:37.652759+0000","last_undegraded":"2024-05-09T07:39:37.652759+0000","last_fullsized":"2024-05-09T07:39:37.652759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:14:26.870086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2520","reported_seq":2556,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.722753+0000","last_change":"2024-05-09T07:13:45.485877+0000","last_active":"2024-05-09T07:39:37.722753+0000","last_peered":"2024-05-09T07:39:37.722753+0000","last_clean":"2024-05-09T07:39:37.722753+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:39:37.722753+0000","last_undegraded":"2024-05-09T07:39:37.722753+0000","last_fullsized":"2024-05-09T07:39:37.722753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:09.783544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2550","reported_seq":2585,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.690064+0000","last_change":"2024-05-09T07:13:45.492643+0000","last_active":"2024-05-09T07:39:37.690064+0000","last_peered":"2024-05-09T07:39:37.690064+0000","last_clean":"2024-05-09T07:39:37.690064+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:39:37.690064+0000","last_undegraded":"2024-05-09T07:39:37.690064+0000","last_fullsized":"2024-05-09T07:39:37.690064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:19:37.261260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2592","reported_seq":2630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.713701+0000","last_change":"2024-05-09T07:13:45.484778+0000","last_active":"2024-05-09T07:39:37.713701+0000","last_peered":"2024-05-09T07:39:37.713701+0000","last_clean":"2024-05-09T07:39:37.713701+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:39:37.713701+0000","last_undegraded":"2024-05-09T07:39:37.713701+0000","last_fullsized":"2024-05-09T07:39:37.713701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2671","reported_seq":2708,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.653817+0000","last_change":"2024-05-09T07:13:45.469339+0000","last_active":"2024-05-09T07:39:37.653817+0000","last_peered":"2024-05-09T07:39:37.653817+0000","last_clean":"2024-05-09T07:39:37.653817+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:39:37.653817+0000","last_undegraded":"2024-05-09T07:39:37.653817+0000","last_fullsized":"2024-05-09T07:39:37.653817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2671,"log_dups_size":0,"ondisk_log_size":2671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:57:20.412941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2502","reported_seq":2540,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.676538+0000","last_change":"2024-05-09T07:13:45.475621+0000","last_active":"2024-05-09T07:39:37.676538+0000","last_peered":"2024-05-09T07:39:37.676538+0000","last_clean":"2024-05-09T07:39:37.676538+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:39:37.676538+0000","last_undegraded":"2024-05-09T07:39:37.676538+0000","last_fullsized":"2024-05-09T07:39:37.676538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2502,"log_dups_size":0,"ondisk_log_size":2502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:59:16.158510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2487","reported_seq":2524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.703825+0000","last_change":"2024-05-09T07:13:45.493591+0000","last_active":"2024-05-09T07:39:37.703825+0000","last_peered":"2024-05-09T07:39:37.703825+0000","last_clean":"2024-05-09T07:39:37.703825+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:39:37.703825+0000","last_undegraded":"2024-05-09T07:39:37.703825+0000","last_fullsized":"2024-05-09T07:39:37.703825+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:44.448158+0000","last_clean_scrub_stamp":"2024-05-09T07:13:44.448158+0000","objects_scrubbed":0,"log_size":2487,"log_dups_size":0,"ondisk_log_size":2487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:22:06.340430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12270,"num_read_kb":3117350,"num_write":214738,"num_write_kb":30722215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232137,"ondisk_log_size":232137,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164829,"ondisk_log_size":164829,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":248708190,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":3182,"num_read_kb":19440,"num_write":211794,"num_write_kb":1695582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497475584,"data_stored":497416380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":192940,"ondisk_log_size":192940,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575164,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":960860,"kb_used_data":55124,"kb_used_omap":1,"kb_used_meta":905726,"kb_avail":93410980,"statfs":{"total":96636764160,"available":95652843520,"internally_reserved":0,"allocated":56446976,"data_stored":56268948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":927463882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575163,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104412,"kb_used_data":66836,"kb_used_omap":1,"kb_used_meta":1037566,"kb_avail":93267428,"statfs":{"total":96636764160,"available":95505846272,"internally_reserved":0,"allocated":68440064,"data_stored":68261582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1062468038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575163,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":983016,"kb_used_data":45344,"kb_used_omap":1,"kb_used_meta":937662,"kb_avail":93388824,"statfs":{"total":96636764160,"available":95630155776,"internally_reserved":0,"allocated":46432256,"data_stored":46258861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960166341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575162,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1043696,"kb_used_data":65832,"kb_used_omap":1,"kb_used_meta":977854,"kb_avail":93328144,"statfs":{"total":96636764160,"available":95568019456,"internally_reserved":0,"allocated":67411968,"data_stored":67230417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001322950},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575164,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1220404,"kb_used_data":70316,"kb_used_omap":1,"kb_used_meta":1150078,"kb_avail":93151436,"statfs":{"total":96636764160,"available":95387070464,"internally_reserved":0,"allocated":72003584,"data_stored":71825848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1177680327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575165,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706740,"kb_used_data":53740,"kb_used_omap":1,"kb_used_meta":652990,"kb_avail":93665100,"statfs":{"total":96636764160,"available":95913062400,"internally_reserved":0,"allocated":55029760,"data_stored":54859544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668662213},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575166,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1233900,"kb_used_data":65828,"kb_used_omap":1,"kb_used_meta":1168062,"kb_avail":93137940,"statfs":{"total":96636764160,"available":95373250560,"internally_reserved":0,"allocated":67407872,"data_stored":67230451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1196095941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575164,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":899828,"kb_used_data":66412,"kb_used_omap":1,"kb_used_meta":833406,"kb_avail":93472012,"statfs":{"total":96636764160,"available":95715340288,"internally_reserved":0,"allocated":68005888,"data_stored":67821297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":853408197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67109502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54738944,"data_stored":54738117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71524352,"data_stored":71516005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46141440,"data_stored":46137434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":68149248,"data_stored":68140155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":55562240,"data_stored":55557153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:39:44.514 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.514 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.514 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.514 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.514 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.515 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.516 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.517 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.518 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-09T07:13:33.304098+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=33, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.519 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.520 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.521 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.522 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.523 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-05-09T07:13:43.444485+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=43, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.524 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.525 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.526 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.527 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.528 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.528 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.528 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-05-09T07:13:44.448158+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=44, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:39:44.528 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T07:40:04.529 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:40:04.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.620+0000 7f9c453ce640 1 Processor -- start 2024-05-09T07:40:04.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.620+0000 7f9c453ce640 1 -- start start 2024-05-09T07:40:04.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c453ce640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f9c400be320 0x7f9c400c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c453ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 0x7f9c40155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c453ce640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9c40159840 con 0x7f9c400c5610 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c453ce640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f9c40159310 con 0x7f9c400be320 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c453ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9c40159560 con 0x7f9c400c8b90 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c3effd640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f9c400c5610 0x7f9c40155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:49352/0 (socket says 172.21.15.160:49352) 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c3effd640 1 -- 172.21.15.160:0/1089067611 learned_addr learned my addr 172.21.15.160:0/1089067611 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1626420536 0 0) 0x7f9c40159840 con 0x7f9c400c5610 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9c28003520 con 0x7f9c400c5610 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c3f7fe640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 0x7f9c40155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.621+0000 7f9c3e7fc640 1 --2- 172.21.15.160:0/1089067611 >> v2:172.21.15.188:3300/0 conn(0x7f9c400be320 0x7f9c400c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 302012151 0 0) 0x7f9c28003520 con 0x7f9c400c5610 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 msgr2=0x7f9c40155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 0x7f9c40155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.636 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 >> v2:172.21.15.188:3300/0 conn(0x7f9c400be320 msgr2=0x7f9c400c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 --2- 172.21.15.160:0/1089067611 >> v2:172.21.15.188:3300/0 conn(0x7f9c400be320 0x7f9c400c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c400c77d0 con 0x7f9c400c5610 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f9c30001670 con 0x7f9c400c5610 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 >> v1:172.21.15.160:6789/0 conn(0x7f9c400c5610 legacy=0x7f9c40155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 0x7f9c28004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c28004d40 0x7f9c280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40159310 con 0x7f9c280039b0 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9c40159560 con 0x7f9c28004d40 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9c2800ba40 con 0x7f9c400c8b90 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3effd640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3f7fe640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c28004d40 0x7f9c280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.637 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3e7fc640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 0x7f9c28004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3effd640 1 -- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 msgr2=0x7f9c28004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3effd640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400c8b90 0x7f9c28004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3effd640 1 -- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c28004d40 msgr2=0x7f9c280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3effd640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c28004d40 0x7f9c280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.622+0000 7f9c3effd640 1 -- 172.21.15.160:0/1089067611 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c28003520 con 0x7f9c280039b0 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.623+0000 7f9c3effd640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c30007fd0 tx=0x7f9c28005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cc93bda6c009af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.623+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30007d50 con 0x7f9c280039b0 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.623+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c30005ad0 con 0x7f9c280039b0 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.623+0000 7f9c3dffb640 1 -- 172.21.15.160:0/1089067611 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30007070 con 0x7f9c280039b0 2024-05-09T07:40:04.638 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c453ce640 1 -- 172.21.15.160:0/1089067611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 msgr2=0x7f9c28003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c453ce640 1 --2- 172.21.15.160:0/1089067611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c28003dd0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9c30007fd0 tx=0x7f9c28005830 comp rx=0 tx=0).stop 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c3f7fe640 1 -- 172.21.15.160:0/1089067611 reap_dead start 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c453ce640 1 -- 172.21.15.160:0/1089067611 shutdown_connections 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c453ce640 1 -- 172.21.15.160:0/1089067611 >> 172.21.15.160:0/1089067611 conn(0x7f9c40072e90 msgr2=0x7f9c400ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c453ce640 1 -- 172.21.15.160:0/1089067611 shutdown_connections 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.624+0000 7f9c453ce640 1 -- 172.21.15.160:0/1089067611 wait complete. 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.625+0000 7f9c453ce640 1 Processor -- start 2024-05-09T07:40:04.639 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.625+0000 7f9c453ce640 1 -- start start 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c400c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400bfc40 0x7f9c400c02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c400c4c80 0x7f9c400c07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400ad6a0 con 0x7f9c280039b0 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f9c400c7d90 con 0x7f9c400c4c80 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f9c40152bb0 con 0x7f9c400bfc40 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c400c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c400c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:41586/0 (socket says 172.21.15.160:41586) 2024-05-09T07:40:04.642 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 -- 172.21.15.160:0/3367679949 learned_addr learned my addr 172.21.15.160:0/3367679949 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3e7fc640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400bfc40 0x7f9c400c02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3f7fe640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c400c4c80 0x7f9c400c07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 -- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400bfc40 msgr2=0x7f9c400c02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400bfc40 0x7f9c400c02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 -- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c400c4c80 msgr2=0x7f9c400c07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c400c4c80 0x7f9c400c07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 -- 172.21.15.160:0/3367679949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c401e23b0 con 0x7f9c280039b0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c3effd640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c400c3a00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c300 tx=0x7f9c400c6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30042070 con 0x7f9c280039b0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9c30007480 con 0x7f9c280039b0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c30001cb0 con 0x7f9c280039b0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.626+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c401ef580 con 0x7f9c280039b0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.627+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c401ef2a0 con 0x7f9c280039b0 2024-05-09T07:40:04.643 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.627+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c400756c0 con 0x7f9c280039b0 2024-05-09T07:40:04.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.628+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f9c30045070 con 0x7f9c280039b0 2024-05-09T07:40:04.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.629+0000 7f9c277fe640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9c2c06be60 0x7f9c2c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:04.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.629+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f9c300abfa0 con 0x7f9c280039b0 2024-05-09T07:40:04.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.629+0000 7f9c3e7fc640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9c2c06be60 0x7f9c2c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:04.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.629+0000 7f9c3e7fc640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9c2c06be60 0x7f9c2c06e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c400c0040 tx=0x7f9c34009190 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:04.644 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.632+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9c3004c030 con 0x7f9c280039b0 2024-05-09T07:40:04.820 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.809+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c400cb100 con 0x7f9c2c06be60 2024-05-09T07:40:04.828 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.817+0000 7f9c277fe640 1 -- 172.21.15.160:0/3367679949 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336364 (secure 0 0 0) 0x7f9c400cb100 con 0x7f9c2c06be60 2024-05-09T07:40:04.832 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:40:04.834 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.820+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9c2c06be60 msgr2=0x7f9c2c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.820+0000 7f9c453ce640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9c2c06be60 0x7f9c2c06e320 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9c400c0040 tx=0x7f9c34009190 comp rx=0 tx=0).stop 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 msgr2=0x7f9c400c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c400c3a00 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c300 tx=0x7f9c400c6520 comp rx=0 tx=0).stop 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 shutdown_connections 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f9c2c06be60 0x7f9c2c06e320 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f9c400c4c80 0x7f9c400c07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.836 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f9c400bfc40 0x7f9c400c02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 --2- 172.21.15.160:0/3367679949 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c280039b0 0x7f9c400c3a00 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:04.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 >> 172.21.15.160:0/3367679949 conn(0x7f9c40072e90 msgr2=0x7f9c400aad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:04.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 shutdown_connections 2024-05-09T07:40:04.837 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:04.821+0000 7f9c453ce640 1 -- 172.21.15.160:0/3367679949 wait complete. 2024-05-09T07:40:04.843 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":815,"stamp":"2024-05-09T07:40:03.567211+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":249305277,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":15568,"num_read_kb":3136890,"num_write":427814,"num_write_kb":32420221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9301,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":590100,"ondisk_log_size":590100,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8156848,"kb_used_data":489264,"kb_used_omap":12,"kb_used_meta":7667507,"kb_avail":746817872,"statfs":{"total":773094113280,"available":764741500928,"internally_reserved":0,"allocated":501006336,"data_stored":499581970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":7851527729},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002655"},"pg_stats":[{"pgid":"2.1f","version":"27'2677","reported_seq":2725,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.638587+0000","last_change":"2024-05-09T07:39:43.638613+0000","last_active":"2024-05-09T07:39:43.638587+0000","last_peered":"2024-05-09T07:39:43.638587+0000","last_clean":"2024-05-09T07:39:43.638587+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:39:43.638587+0000","last_undegraded":"2024-05-09T07:39:43.638587+0000","last_fullsized":"2024-05-09T07:39:43.638587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2677","last_scrub_stamp":"2024-05-09T07:39:43.638526+0000","last_deep_scrub":"27'2677","last_deep_scrub_stamp":"2024-05-09T07:39:43.638526+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.638526+0000","objects_scrubbed":2,"log_size":2677,"log_dups_size":0,"ondisk_log_size":2677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:03.343390+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":2687,"num_write_kb":18241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":561,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2578","reported_seq":2626,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.229724+0000","last_change":"2024-05-09T07:39:46.229724+0000","last_active":"2024-05-09T07:39:46.229724+0000","last_peered":"2024-05-09T07:39:46.229724+0000","last_clean":"2024-05-09T07:39:46.229724+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:39:46.229724+0000","last_undegraded":"2024-05-09T07:39:46.229724+0000","last_fullsized":"2024-05-09T07:39:46.229724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2578","last_scrub_stamp":"2024-05-09T07:39:46.229628+0000","last_deep_scrub":"27'2578","last_deep_scrub_stamp":"2024-05-09T07:39:46.229628+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.229628+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:51.406925+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9823","reported_seq":10254,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.600358+0000","last_change":"2024-05-09T07:13:49.940963+0000","last_active":"2024-05-09T07:39:34.600358+0000","last_peered":"2024-05-09T07:39:34.600358+0000","last_clean":"2024-05-09T07:39:34.600358+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:39:34.600358+0000","last_undegraded":"2024-05-09T07:39:34.600358+0000","last_fullsized":"2024-05-09T07:39:34.600358+0000","mapping_epoch":22,"log_start":"27'3504","ondisk_log_start":"27'3504","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6319,"log_dups_size":0,"ondisk_log_size":6319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":399,"num_read_kb":101346,"num_write":7176,"num_write_kb":1009567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'7790","reported_seq":7840,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.239196+0000","last_change":"2024-05-09T07:39:48.239196+0000","last_active":"2024-05-09T07:39:48.239196+0000","last_peered":"2024-05-09T07:39:48.239196+0000","last_clean":"2024-05-09T07:39:48.239196+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:39:48.239196+0000","last_undegraded":"2024-05-09T07:39:48.239196+0000","last_fullsized":"2024-05-09T07:39:48.239196+0000","mapping_epoch":17,"log_start":"27'1500","ondisk_log_start":"27'1500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7790","last_scrub_stamp":"2024-05-09T07:39:48.239088+0000","last_deep_scrub":"27'7790","last_deep_scrub_stamp":"2024-05-09T07:39:48.239088+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.239088+0000","objects_scrubbed":1,"log_size":6290,"log_dups_size":0,"ondisk_log_size":6290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.345864+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":7802,"num_write_kb":54453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2558","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.586739+0000","last_change":"2024-05-09T07:39:50.586762+0000","last_active":"2024-05-09T07:39:50.586739+0000","last_peered":"2024-05-09T07:39:50.586739+0000","last_clean":"2024-05-09T07:39:50.586739+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:39:50.586739+0000","last_undegraded":"2024-05-09T07:39:50.586739+0000","last_fullsized":"2024-05-09T07:39:50.586739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-05-09T07:39:50.586658+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-05-09T07:39:50.586658+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.586658+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:36.419009+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9533","reported_seq":9929,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.351870+0000","last_change":"2024-05-09T07:39:50.351903+0000","last_active":"2024-05-09T07:39:50.351870+0000","last_peered":"2024-05-09T07:39:50.351870+0000","last_clean":"2024-05-09T07:39:50.351870+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:39:50.351870+0000","last_undegraded":"2024-05-09T07:39:50.351870+0000","last_fullsized":"2024-05-09T07:39:50.351870+0000","mapping_epoch":22,"log_start":"27'2329","ondisk_log_start":"27'2329","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9533","last_scrub_stamp":"2024-05-09T07:39:50.351773+0000","last_deep_scrub":"27'9533","last_deep_scrub_stamp":"2024-05-09T07:39:50.351773+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.351773+0000","objects_scrubbed":0,"log_size":7204,"log_dups_size":0,"ondisk_log_size":7204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:41.214574+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6761,"num_write_kb":1095702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9533","reported_seq":9720,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.436111+0000","last_change":"2024-05-09T07:39:46.436150+0000","last_active":"2024-05-09T07:39:46.436111+0000","last_peered":"2024-05-09T07:39:46.436111+0000","last_clean":"2024-05-09T07:39:46.436111+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:39:46.436111+0000","last_undegraded":"2024-05-09T07:39:46.436111+0000","last_fullsized":"2024-05-09T07:39:46.436111+0000","mapping_epoch":17,"log_start":"27'2500","ondisk_log_start":"27'2500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9533","last_scrub_stamp":"2024-05-09T07:39:46.436028+0000","last_deep_scrub":"27'9533","last_deep_scrub_stamp":"2024-05-09T07:39:46.436028+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.436028+0000","objects_scrubbed":3,"log_size":7033,"log_dups_size":0,"ondisk_log_size":7033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:53.364308+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5224923,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":1099,"num_write":9529,"num_write_kb":77058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2666","reported_seq":2711,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.987062+0000","last_change":"2024-05-09T07:39:55.987094+0000","last_active":"2024-05-09T07:39:55.987062+0000","last_peered":"2024-05-09T07:39:55.987062+0000","last_clean":"2024-05-09T07:39:55.987062+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:39:55.987062+0000","last_undegraded":"2024-05-09T07:39:55.987062+0000","last_fullsized":"2024-05-09T07:39:55.987062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2666","last_scrub_stamp":"2024-05-09T07:39:55.986977+0000","last_deep_scrub":"27'2666","last_deep_scrub_stamp":"2024-05-09T07:39:55.986977+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.986977+0000","objects_scrubbed":0,"log_size":2666,"log_dups_size":0,"ondisk_log_size":2666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:19.197944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9072","reported_seq":9490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.301275+0000","last_change":"2024-05-09T07:39:53.301303+0000","last_active":"2024-05-09T07:39:53.301275+0000","last_peered":"2024-05-09T07:39:53.301275+0000","last_clean":"2024-05-09T07:39:53.301275+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:39:53.301275+0000","last_undegraded":"2024-05-09T07:39:53.301275+0000","last_fullsized":"2024-05-09T07:39:53.301275+0000","mapping_epoch":22,"log_start":"27'1901","ondisk_log_start":"27'1901","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9072","last_scrub_stamp":"2024-05-09T07:39:53.301167+0000","last_deep_scrub":"27'9072","last_deep_scrub_stamp":"2024-05-09T07:39:53.301167+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.301167+0000","objects_scrubbed":0,"log_size":7171,"log_dups_size":0,"ondisk_log_size":7171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:09.641382+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6307,"num_write_kb":900735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4294","reported_seq":4340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.294687+0000","last_change":"2024-05-09T07:39:51.294742+0000","last_active":"2024-05-09T07:39:51.294687+0000","last_peered":"2024-05-09T07:39:51.294687+0000","last_clean":"2024-05-09T07:39:51.294687+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:39:51.294687+0000","last_undegraded":"2024-05-09T07:39:51.294687+0000","last_fullsized":"2024-05-09T07:39:51.294687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4294","last_scrub_stamp":"2024-05-09T07:39:51.294570+0000","last_deep_scrub":"27'4294","last_deep_scrub_stamp":"2024-05-09T07:39:51.294570+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.294570+0000","objects_scrubbed":2,"log_size":4294,"log_dups_size":0,"ondisk_log_size":4294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:21.037947+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":4306,"num_write_kb":47774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2602","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.062040+0000","last_change":"2024-05-09T07:39:45.062063+0000","last_active":"2024-05-09T07:39:45.062040+0000","last_peered":"2024-05-09T07:39:45.062040+0000","last_clean":"2024-05-09T07:39:45.062040+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:39:45.062040+0000","last_undegraded":"2024-05-09T07:39:45.062040+0000","last_fullsized":"2024-05-09T07:39:45.062040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2602","last_scrub_stamp":"2024-05-09T07:39:45.061935+0000","last_deep_scrub":"27'2602","last_deep_scrub_stamp":"2024-05-09T07:39:45.061935+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.061935+0000","objects_scrubbed":0,"log_size":2602,"log_dups_size":0,"ondisk_log_size":2602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:51.990215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'9250","reported_seq":9732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.357996+0000","last_change":"2024-05-09T07:39:52.358022+0000","last_active":"2024-05-09T07:39:52.357996+0000","last_peered":"2024-05-09T07:39:52.357996+0000","last_clean":"2024-05-09T07:39:52.357996+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:39:52.357996+0000","last_undegraded":"2024-05-09T07:39:52.357996+0000","last_fullsized":"2024-05-09T07:39:52.357996+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9250","last_scrub_stamp":"2024-05-09T07:39:52.357879+0000","last_deep_scrub":"27'9250","last_deep_scrub_stamp":"2024-05-09T07:39:52.357879+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.357879+0000","objects_scrubbed":0,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:38.190231+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":442,"num_read_kb":112268,"num_write":6539,"num_write_kb":853848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2585","reported_seq":2631,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.856131+0000","last_change":"2024-05-09T07:39:50.856177+0000","last_active":"2024-05-09T07:39:50.856131+0000","last_peered":"2024-05-09T07:39:50.856131+0000","last_clean":"2024-05-09T07:39:50.856131+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:39:50.856131+0000","last_undegraded":"2024-05-09T07:39:50.856131+0000","last_fullsized":"2024-05-09T07:39:50.856131+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2585","last_scrub_stamp":"2024-05-09T07:39:50.856027+0000","last_deep_scrub":"27'2585","last_deep_scrub_stamp":"2024-05-09T07:39:50.856027+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.856027+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:49.201658+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6970","reported_seq":7017,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.419458+0000","last_change":"2024-05-09T07:39:47.419494+0000","last_active":"2024-05-09T07:39:47.419458+0000","last_peered":"2024-05-09T07:39:47.419458+0000","last_clean":"2024-05-09T07:39:47.419458+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:39:47.419458+0000","last_undegraded":"2024-05-09T07:39:47.419458+0000","last_fullsized":"2024-05-09T07:39:47.419458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6970","last_scrub_stamp":"2024-05-09T07:39:47.419366+0000","last_deep_scrub":"27'6970","last_deep_scrub_stamp":"2024-05-09T07:39:47.419366+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.419366+0000","objects_scrubbed":4,"log_size":6970,"log_dups_size":0,"ondisk_log_size":6970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:24.872236+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6974,"num_write_kb":53856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":334,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9808","reported_seq":10065,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.331939+0000","last_change":"2024-05-09T07:39:49.331965+0000","last_active":"2024-05-09T07:39:49.331939+0000","last_peered":"2024-05-09T07:39:49.331939+0000","last_clean":"2024-05-09T07:39:49.331939+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:39:49.331939+0000","last_undegraded":"2024-05-09T07:39:49.331939+0000","last_fullsized":"2024-05-09T07:39:49.331939+0000","mapping_epoch":22,"log_start":"27'2633","ondisk_log_start":"27'2633","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9808","last_scrub_stamp":"2024-05-09T07:39:49.331826+0000","last_deep_scrub":"27'9808","last_deep_scrub_stamp":"2024-05-09T07:39:49.331826+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.331826+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:54.509122+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":215,"num_read_kb":54610,"num_write":7041,"num_write_kb":1008697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2616","reported_seq":2662,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.249456+0000","last_change":"2024-05-09T07:39:51.249479+0000","last_active":"2024-05-09T07:39:51.249456+0000","last_peered":"2024-05-09T07:39:51.249456+0000","last_clean":"2024-05-09T07:39:51.249456+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:39:51.249456+0000","last_undegraded":"2024-05-09T07:39:51.249456+0000","last_fullsized":"2024-05-09T07:39:51.249456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2616","last_scrub_stamp":"2024-05-09T07:39:51.249375+0000","last_deep_scrub":"27'2616","last_deep_scrub_stamp":"2024-05-09T07:39:51.249375+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.249375+0000","objects_scrubbed":0,"log_size":2616,"log_dups_size":0,"ondisk_log_size":2616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:16.871100+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7784","reported_seq":7833,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.348520+0000","last_change":"2024-05-09T07:39:43.348520+0000","last_active":"2024-05-09T07:39:43.348520+0000","last_peered":"2024-05-09T07:39:43.348520+0000","last_clean":"2024-05-09T07:39:43.348520+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:39:43.348520+0000","last_undegraded":"2024-05-09T07:39:43.348520+0000","last_fullsized":"2024-05-09T07:39:43.348520+0000","mapping_epoch":17,"log_start":"27'400","ondisk_log_start":"27'400","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7784","last_scrub_stamp":"2024-05-09T07:39:43.348350+0000","last_deep_scrub":"27'7784","last_deep_scrub_stamp":"2024-05-09T07:39:43.348350+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.348350+0000","objects_scrubbed":2,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:49.480366+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":58435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9632","reported_seq":10181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.234971+0000","last_change":"2024-05-09T07:39:55.234999+0000","last_active":"2024-05-09T07:39:55.234971+0000","last_peered":"2024-05-09T07:39:55.234971+0000","last_clean":"2024-05-09T07:39:55.234971+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:39:55.234971+0000","last_undegraded":"2024-05-09T07:39:55.234971+0000","last_fullsized":"2024-05-09T07:39:55.234971+0000","mapping_epoch":22,"log_start":"27'2443","ondisk_log_start":"27'2443","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9632","last_scrub_stamp":"2024-05-09T07:39:55.234858+0000","last_deep_scrub":"27'9632","last_deep_scrub_stamp":"2024-05-09T07:39:55.234858+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.234858+0000","objects_scrubbed":0,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:31.851464+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":508,"num_read_kb":129802,"num_write":6852,"num_write_kb":970185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2685","reported_seq":2732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.276740+0000","last_change":"2024-05-09T07:39:53.276740+0000","last_active":"2024-05-09T07:39:53.276740+0000","last_peered":"2024-05-09T07:39:53.276740+0000","last_clean":"2024-05-09T07:39:53.276740+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:39:53.276740+0000","last_undegraded":"2024-05-09T07:39:53.276740+0000","last_fullsized":"2024-05-09T07:39:53.276740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2685","last_scrub_stamp":"2024-05-09T07:39:53.276635+0000","last_deep_scrub":"27'2685","last_deep_scrub_stamp":"2024-05-09T07:39:53.276635+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.276635+0000","objects_scrubbed":0,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:16.487602+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3775","reported_seq":3824,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.347239+0000","last_change":"2024-05-09T07:39:47.347239+0000","last_active":"2024-05-09T07:39:47.347239+0000","last_peered":"2024-05-09T07:39:47.347239+0000","last_clean":"2024-05-09T07:39:47.347239+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:39:47.347239+0000","last_undegraded":"2024-05-09T07:39:47.347239+0000","last_fullsized":"2024-05-09T07:39:47.347239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3775","last_scrub_stamp":"2024-05-09T07:39:47.347152+0000","last_deep_scrub":"27'3775","last_deep_scrub_stamp":"2024-05-09T07:39:47.347152+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.347152+0000","objects_scrubbed":0,"log_size":3775,"log_dups_size":0,"ondisk_log_size":3775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:45.692783+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3772,"num_write_kb":27117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9053","reported_seq":9674,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.780347+0000","last_change":"2024-05-09T07:39:52.780393+0000","last_active":"2024-05-09T07:39:52.780347+0000","last_peered":"2024-05-09T07:39:52.780347+0000","last_clean":"2024-05-09T07:39:52.780347+0000","last_became_active":"2024-05-09T07:14:26.646222+0000","last_became_peered":"2024-05-09T07:14:26.646222+0000","last_unstale":"2024-05-09T07:39:52.780347+0000","last_undegraded":"2024-05-09T07:39:52.780347+0000","last_fullsized":"2024-05-09T07:39:52.780347+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9053","last_scrub_stamp":"2024-05-09T07:39:52.780216+0000","last_deep_scrub":"27'9053","last_deep_scrub_stamp":"2024-05-09T07:39:52.780216+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.780216+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:59.708496+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6377,"num_write_kb":893211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2588","reported_seq":2633,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.267234+0000","last_change":"2024-05-09T07:39:46.267264+0000","last_active":"2024-05-09T07:39:46.267234+0000","last_peered":"2024-05-09T07:39:46.267234+0000","last_clean":"2024-05-09T07:39:46.267234+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:39:46.267234+0000","last_undegraded":"2024-05-09T07:39:46.267234+0000","last_fullsized":"2024-05-09T07:39:46.267234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2588","last_scrub_stamp":"2024-05-09T07:39:46.267114+0000","last_deep_scrub":"27'2588","last_deep_scrub_stamp":"2024-05-09T07:39:46.267114+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.267114+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:22.118506+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2268","reported_seq":2316,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.298732+0000","last_change":"2024-05-09T07:39:49.298732+0000","last_active":"2024-05-09T07:39:49.298732+0000","last_peered":"2024-05-09T07:39:49.298732+0000","last_clean":"2024-05-09T07:39:49.298732+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:39:49.298732+0000","last_undegraded":"2024-05-09T07:39:49.298732+0000","last_fullsized":"2024-05-09T07:39:49.298732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2268","last_scrub_stamp":"2024-05-09T07:39:49.298603+0000","last_deep_scrub":"27'2268","last_deep_scrub_stamp":"2024-05-09T07:39:49.298603+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.298603+0000","objects_scrubbed":1,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:30.124305+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2276,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9931","reported_seq":10076,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.276205+0000","last_change":"2024-05-09T07:39:54.276230+0000","last_active":"2024-05-09T07:39:54.276205+0000","last_peered":"2024-05-09T07:39:54.276205+0000","last_clean":"2024-05-09T07:39:54.276205+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:39:54.276205+0000","last_undegraded":"2024-05-09T07:39:54.276205+0000","last_fullsized":"2024-05-09T07:39:54.276205+0000","mapping_epoch":22,"log_start":"27'2728","ondisk_log_start":"27'2728","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9931","last_scrub_stamp":"2024-05-09T07:39:54.276122+0000","last_deep_scrub":"27'9931","last_deep_scrub_stamp":"2024-05-09T07:39:54.276122+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.276122+0000","objects_scrubbed":0,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:51.582863+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":26416,"num_write":7236,"num_write_kb":974347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2542","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.617914+0000","last_change":"2024-05-09T07:39:49.617914+0000","last_active":"2024-05-09T07:39:49.617914+0000","last_peered":"2024-05-09T07:39:49.617914+0000","last_clean":"2024-05-09T07:39:49.617914+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:39:49.617914+0000","last_undegraded":"2024-05-09T07:39:49.617914+0000","last_fullsized":"2024-05-09T07:39:49.617914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2542","last_scrub_stamp":"2024-05-09T07:39:49.617750+0000","last_deep_scrub":"27'2542","last_deep_scrub_stamp":"2024-05-09T07:39:49.617750+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.617750+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:07.724527+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4070","reported_seq":4119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.940913+0000","last_change":"2024-05-09T07:39:47.940984+0000","last_active":"2024-05-09T07:39:47.940913+0000","last_peered":"2024-05-09T07:39:47.940913+0000","last_clean":"2024-05-09T07:39:47.940913+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:39:47.940913+0000","last_undegraded":"2024-05-09T07:39:47.940913+0000","last_fullsized":"2024-05-09T07:39:47.940913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4070","last_scrub_stamp":"2024-05-09T07:39:47.940825+0000","last_deep_scrub":"27'4070","last_deep_scrub_stamp":"2024-05-09T07:39:47.940825+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.940825+0000","objects_scrubbed":4,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:07.645688+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":39474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9249","reported_seq":9676,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.265154+0000","last_change":"2024-05-09T07:39:56.265183+0000","last_active":"2024-05-09T07:39:56.265154+0000","last_peered":"2024-05-09T07:39:56.265154+0000","last_clean":"2024-05-09T07:39:56.265154+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:39:56.265154+0000","last_undegraded":"2024-05-09T07:39:56.265154+0000","last_fullsized":"2024-05-09T07:39:56.265154+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9249","last_scrub_stamp":"2024-05-09T07:39:56.265056+0000","last_deep_scrub":"27'9249","last_deep_scrub_stamp":"2024-05-09T07:39:56.265056+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.265056+0000","objects_scrubbed":0,"log_size":7227,"log_dups_size":0,"ondisk_log_size":7227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:19.476023+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":97790,"num_write":6537,"num_write_kb":843792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2541","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.300176+0000","last_change":"2024-05-09T07:39:56.300199+0000","last_active":"2024-05-09T07:39:56.300176+0000","last_peered":"2024-05-09T07:39:56.300176+0000","last_clean":"2024-05-09T07:39:56.300176+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:39:56.300176+0000","last_undegraded":"2024-05-09T07:39:56.300176+0000","last_fullsized":"2024-05-09T07:39:56.300176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2541","last_scrub_stamp":"2024-05-09T07:39:56.300098+0000","last_deep_scrub":"27'2541","last_deep_scrub_stamp":"2024-05-09T07:39:56.300098+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.300098+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:27.151559+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6998","reported_seq":7654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.402653+0000","last_change":"2024-05-09T07:39:42.402653+0000","last_active":"2024-05-09T07:39:42.402653+0000","last_peered":"2024-05-09T07:39:42.402653+0000","last_clean":"2024-05-09T07:39:42.402653+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:39:42.402653+0000","last_undegraded":"2024-05-09T07:39:42.402653+0000","last_fullsized":"2024-05-09T07:39:42.402653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6998","last_scrub_stamp":"2024-05-09T07:39:42.402522+0000","last_deep_scrub":"27'6998","last_deep_scrub_stamp":"2024-05-09T07:39:42.402522+0000","last_clean_scrub_stamp":"2024-05-09T07:39:42.402522+0000","objects_scrubbed":3,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:53.052513+0000","scrub_duration":85,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":608,"num_read_kb":4487,"num_write":9817,"num_write_kb":61142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9154","reported_seq":9338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.674527+0000","last_change":"2024-05-09T07:13:49.939272+0000","last_active":"2024-05-09T07:39:34.674527+0000","last_peered":"2024-05-09T07:39:34.674527+0000","last_clean":"2024-05-09T07:39:34.674527+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:39:34.674527+0000","last_undegraded":"2024-05-09T07:39:34.674527+0000","last_fullsized":"2024-05-09T07:39:34.674527+0000","mapping_epoch":22,"log_start":"27'2834","ondisk_log_start":"27'2834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":38100,"num_write":6430,"num_write_kb":884596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2640","reported_seq":2687,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.354485+0000","last_change":"2024-05-09T07:39:52.354509+0000","last_active":"2024-05-09T07:39:52.354485+0000","last_peered":"2024-05-09T07:39:52.354485+0000","last_clean":"2024-05-09T07:39:52.354485+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:39:52.354485+0000","last_undegraded":"2024-05-09T07:39:52.354485+0000","last_fullsized":"2024-05-09T07:39:52.354485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2640","last_scrub_stamp":"2024-05-09T07:39:52.354398+0000","last_deep_scrub":"27'2640","last_deep_scrub_stamp":"2024-05-09T07:39:52.354398+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.354398+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:18.855399+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6526","reported_seq":6574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.358383+0000","last_change":"2024-05-09T07:39:48.358414+0000","last_active":"2024-05-09T07:39:48.358383+0000","last_peered":"2024-05-09T07:39:48.358383+0000","last_clean":"2024-05-09T07:39:48.358383+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:39:48.358383+0000","last_undegraded":"2024-05-09T07:39:48.358383+0000","last_fullsized":"2024-05-09T07:39:48.358383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6526","last_scrub_stamp":"2024-05-09T07:39:48.358312+0000","last_deep_scrub":"27'6526","last_deep_scrub_stamp":"2024-05-09T07:39:48.358312+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.358312+0000","objects_scrubbed":3,"log_size":6526,"log_dups_size":0,"ondisk_log_size":6526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:24.209704+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6527,"num_write_kb":53434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9270","reported_seq":10165,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.576205+0000","last_change":"2024-05-09T07:39:56.576230+0000","last_active":"2024-05-09T07:39:56.576205+0000","last_peered":"2024-05-09T07:39:56.576205+0000","last_clean":"2024-05-09T07:39:56.576205+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:39:56.576205+0000","last_undegraded":"2024-05-09T07:39:56.576205+0000","last_fullsized":"2024-05-09T07:39:56.576205+0000","mapping_epoch":22,"log_start":"27'1856","ondisk_log_start":"27'1856","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9270","last_scrub_stamp":"2024-05-09T07:39:56.576122+0000","last_deep_scrub":"27'9270","last_deep_scrub_stamp":"2024-05-09T07:39:56.576122+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.576122+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:59.746690+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":853,"num_read_kb":216662,"num_write":6474,"num_write_kb":849337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2527","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.748842+0000","last_change":"2024-05-09T07:39:50.748870+0000","last_active":"2024-05-09T07:39:50.748842+0000","last_peered":"2024-05-09T07:39:50.748842+0000","last_clean":"2024-05-09T07:39:50.748842+0000","last_became_active":"2024-05-09T07:14:26.645530+0000","last_became_peered":"2024-05-09T07:14:26.645530+0000","last_unstale":"2024-05-09T07:39:50.748842+0000","last_undegraded":"2024-05-09T07:39:50.748842+0000","last_fullsized":"2024-05-09T07:39:50.748842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2527","last_scrub_stamp":"2024-05-09T07:39:50.748763+0000","last_deep_scrub":"27'2527","last_deep_scrub_stamp":"2024-05-09T07:39:50.748763+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.748763+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:01.398753+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'8380","reported_seq":8429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.430910+0000","last_change":"2024-05-09T07:39:44.430951+0000","last_active":"2024-05-09T07:39:44.430910+0000","last_peered":"2024-05-09T07:39:44.430910+0000","last_clean":"2024-05-09T07:39:44.430910+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:39:44.430910+0000","last_undegraded":"2024-05-09T07:39:44.430910+0000","last_fullsized":"2024-05-09T07:39:44.430910+0000","mapping_epoch":17,"log_start":"27'1000","ondisk_log_start":"27'1000","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8380","last_scrub_stamp":"2024-05-09T07:39:44.430840+0000","last_deep_scrub":"27'8380","last_deep_scrub_stamp":"2024-05-09T07:39:44.430840+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.430840+0000","objects_scrubbed":6,"log_size":7380,"log_dups_size":0,"ondisk_log_size":7380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:04.135704+0000","scrub_duration":68,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8370,"num_write_kb":75026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3406,"num_omap_keys":6,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9858","reported_seq":10338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.345070+0000","last_change":"2024-05-09T07:39:56.345111+0000","last_active":"2024-05-09T07:39:56.345070+0000","last_peered":"2024-05-09T07:39:56.345070+0000","last_clean":"2024-05-09T07:39:56.345070+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:39:56.345070+0000","last_undegraded":"2024-05-09T07:39:56.345070+0000","last_fullsized":"2024-05-09T07:39:56.345070+0000","mapping_epoch":22,"log_start":"27'2476","ondisk_log_start":"27'2476","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9858","last_scrub_stamp":"2024-05-09T07:39:56.344972+0000","last_deep_scrub":"27'9858","last_deep_scrub_stamp":"2024-05-09T07:39:56.344972+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.344972+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:32.670316+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":438,"num_read_kb":111252,"num_write":7195,"num_write_kb":963969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2574","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.256006+0000","last_change":"2024-05-09T07:39:44.256058+0000","last_active":"2024-05-09T07:39:44.256006+0000","last_peered":"2024-05-09T07:39:44.256006+0000","last_clean":"2024-05-09T07:39:44.256006+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:39:44.256006+0000","last_undegraded":"2024-05-09T07:39:44.256006+0000","last_fullsized":"2024-05-09T07:39:44.256006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2574","last_scrub_stamp":"2024-05-09T07:39:44.255885+0000","last_deep_scrub":"27'2574","last_deep_scrub_stamp":"2024-05-09T07:39:44.255885+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.255885+0000","objects_scrubbed":0,"log_size":2574,"log_dups_size":0,"ondisk_log_size":2574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:50.387900+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'9200","reported_seq":9249,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.130589+0000","last_change":"2024-05-09T07:39:49.130620+0000","last_active":"2024-05-09T07:39:49.130589+0000","last_peered":"2024-05-09T07:39:49.130589+0000","last_clean":"2024-05-09T07:39:49.130589+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:39:49.130589+0000","last_undegraded":"2024-05-09T07:39:49.130589+0000","last_fullsized":"2024-05-09T07:39:49.130589+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9200","last_scrub_stamp":"2024-05-09T07:39:49.130519+0000","last_deep_scrub":"27'9200","last_deep_scrub_stamp":"2024-05-09T07:39:49.130519+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.130519+0000","objects_scrubbed":2,"log_size":7400,"log_dups_size":0,"ondisk_log_size":7400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:54.307815+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9200,"num_write_kb":63563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9948","reported_seq":10122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.353807+0000","last_change":"2024-05-09T07:39:54.353857+0000","last_active":"2024-05-09T07:39:54.353807+0000","last_peered":"2024-05-09T07:39:54.353807+0000","last_clean":"2024-05-09T07:39:54.353807+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:39:54.353807+0000","last_undegraded":"2024-05-09T07:39:54.353807+0000","last_fullsized":"2024-05-09T07:39:54.353807+0000","mapping_epoch":22,"log_start":"27'2616","ondisk_log_start":"27'2616","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9948","last_scrub_stamp":"2024-05-09T07:39:54.353678+0000","last_deep_scrub":"27'9948","last_deep_scrub_stamp":"2024-05-09T07:39:54.353678+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.353678+0000","objects_scrubbed":0,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:30.970283+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":33528,"num_write":7177,"num_write_kb":934133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2581","reported_seq":2629,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.343555+0000","last_change":"2024-05-09T07:39:53.343576+0000","last_active":"2024-05-09T07:39:53.343555+0000","last_peered":"2024-05-09T07:39:53.343555+0000","last_clean":"2024-05-09T07:39:53.343555+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:39:53.343555+0000","last_undegraded":"2024-05-09T07:39:53.343555+0000","last_fullsized":"2024-05-09T07:39:53.343555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2581","last_scrub_stamp":"2024-05-09T07:39:53.343492+0000","last_deep_scrub":"27'2581","last_deep_scrub_stamp":"2024-05-09T07:39:53.343492+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.343492+0000","objects_scrubbed":0,"log_size":2581,"log_dups_size":0,"ondisk_log_size":2581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:50.650233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7809","reported_seq":8757,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.810778+0000","last_change":"2024-05-09T07:39:44.810810+0000","last_active":"2024-05-09T07:39:44.810778+0000","last_peered":"2024-05-09T07:39:44.810778+0000","last_clean":"2024-05-09T07:39:44.810778+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:39:44.810778+0000","last_undegraded":"2024-05-09T07:39:44.810778+0000","last_fullsized":"2024-05-09T07:39:44.810778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7809","last_scrub_stamp":"2024-05-09T07:39:44.810716+0000","last_deep_scrub":"27'7809","last_deep_scrub_stamp":"2024-05-09T07:39:44.810716+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.810716+0000","objects_scrubbed":3,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:30.496241+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8607492,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":900,"num_read_kb":4675,"num_write":7792,"num_write_kb":62664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9564","reported_seq":9949,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.301961+0000","last_change":"2024-05-09T07:39:55.301961+0000","last_active":"2024-05-09T07:39:55.301961+0000","last_peered":"2024-05-09T07:39:55.301961+0000","last_clean":"2024-05-09T07:39:55.301961+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:39:55.301961+0000","last_undegraded":"2024-05-09T07:39:55.301961+0000","last_fullsized":"2024-05-09T07:39:55.301961+0000","mapping_epoch":22,"log_start":"27'2181","ondisk_log_start":"27'2181","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9564","last_scrub_stamp":"2024-05-09T07:39:55.301777+0000","last_deep_scrub":"27'9564","last_deep_scrub_stamp":"2024-05-09T07:39:55.301777+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.301777+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:18.512745+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":87122,"num_write":6858,"num_write_kb":899446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2524","reported_seq":2570,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.676783+0000","last_change":"2024-05-09T07:39:45.676804+0000","last_active":"2024-05-09T07:39:45.676783+0000","last_peered":"2024-05-09T07:39:45.676783+0000","last_clean":"2024-05-09T07:39:45.676783+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:39:45.676783+0000","last_undegraded":"2024-05-09T07:39:45.676783+0000","last_fullsized":"2024-05-09T07:39:45.676783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2524","last_scrub_stamp":"2024-05-09T07:39:45.676715+0000","last_deep_scrub":"27'2524","last_deep_scrub_stamp":"2024-05-09T07:39:45.676715+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.676715+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:13.837357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7853","reported_seq":7902,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.268486+0000","last_change":"2024-05-09T07:39:50.268486+0000","last_active":"2024-05-09T07:39:50.268486+0000","last_peered":"2024-05-09T07:39:50.268486+0000","last_clean":"2024-05-09T07:39:50.268486+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:39:50.268486+0000","last_undegraded":"2024-05-09T07:39:50.268486+0000","last_fullsized":"2024-05-09T07:39:50.268486+0000","mapping_epoch":17,"log_start":"27'800","ondisk_log_start":"27'800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7853","last_scrub_stamp":"2024-05-09T07:39:50.268374+0000","last_deep_scrub":"27'7853","last_deep_scrub_stamp":"2024-05-09T07:39:50.268374+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.268374+0000","objects_scrubbed":1,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:55.445670+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7854,"num_write_kb":58481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9413","reported_seq":9677,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.578326+0000","last_change":"2024-05-09T07:39:55.578351+0000","last_active":"2024-05-09T07:39:55.578326+0000","last_peered":"2024-05-09T07:39:55.578326+0000","last_clean":"2024-05-09T07:39:55.578326+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:39:55.578326+0000","last_undegraded":"2024-05-09T07:39:55.578326+0000","last_fullsized":"2024-05-09T07:39:55.578326+0000","mapping_epoch":22,"log_start":"27'2042","ondisk_log_start":"27'2042","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9413","last_scrub_stamp":"2024-05-09T07:39:55.578237+0000","last_deep_scrub":"27'9413","last_deep_scrub_stamp":"2024-05-09T07:39:55.578237+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.578237+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:51.820544+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":222,"num_read_kb":56388,"num_write":6719,"num_write_kb":1113426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2642","reported_seq":2689,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.220909+0000","last_change":"2024-05-09T07:39:45.220931+0000","last_active":"2024-05-09T07:39:45.220909+0000","last_peered":"2024-05-09T07:39:45.220909+0000","last_clean":"2024-05-09T07:39:45.220909+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:39:45.220909+0000","last_undegraded":"2024-05-09T07:39:45.220909+0000","last_fullsized":"2024-05-09T07:39:45.220909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2642","last_scrub_stamp":"2024-05-09T07:39:45.220837+0000","last_deep_scrub":"27'2642","last_deep_scrub_stamp":"2024-05-09T07:39:45.220837+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.220837+0000","objects_scrubbed":0,"log_size":2642,"log_dups_size":0,"ondisk_log_size":2642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:43.566468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4652","reported_seq":4701,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.108555+0000","last_change":"2024-05-09T07:39:48.108619+0000","last_active":"2024-05-09T07:39:48.108555+0000","last_peered":"2024-05-09T07:39:48.108555+0000","last_clean":"2024-05-09T07:39:48.108555+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:39:48.108555+0000","last_undegraded":"2024-05-09T07:39:48.108555+0000","last_fullsized":"2024-05-09T07:39:48.108555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4652","last_scrub_stamp":"2024-05-09T07:39:48.108460+0000","last_deep_scrub":"27'4652","last_deep_scrub_stamp":"2024-05-09T07:39:48.108460+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.108460+0000","objects_scrubbed":1,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:28.934163+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4655,"num_write_kb":39916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":207,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.417352+0000","last_change":"2024-05-09T07:39:45.417389+0000","last_active":"2024-05-09T07:39:45.417352+0000","last_peered":"2024-05-09T07:39:45.417352+0000","last_clean":"2024-05-09T07:39:45.417352+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:39:45.417352+0000","last_undegraded":"2024-05-09T07:39:45.417352+0000","last_fullsized":"2024-05-09T07:39:45.417352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-05-09T07:39:45.417268+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-05-09T07:39:45.417268+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.417268+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:58.531645+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8970","reported_seq":9599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.849579+0000","last_change":"2024-05-09T07:39:57.849622+0000","last_active":"2024-05-09T07:39:57.849579+0000","last_peered":"2024-05-09T07:39:57.849579+0000","last_clean":"2024-05-09T07:39:57.849579+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:39:57.849579+0000","last_undegraded":"2024-05-09T07:39:57.849579+0000","last_fullsized":"2024-05-09T07:39:57.849579+0000","mapping_epoch":22,"log_start":"27'1027","ondisk_log_start":"27'1027","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8970","last_scrub_stamp":"2024-05-09T07:39:57.849401+0000","last_deep_scrub":"27'8970","last_deep_scrub_stamp":"2024-05-09T07:39:57.849401+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.849401+0000","objects_scrubbed":0,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:08.614258+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":587,"num_read_kb":149098,"num_write":6333,"num_write_kb":900398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2565","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.709609+0000","last_change":"2024-05-09T07:39:49.709651+0000","last_active":"2024-05-09T07:39:49.709609+0000","last_peered":"2024-05-09T07:39:49.709609+0000","last_clean":"2024-05-09T07:39:49.709609+0000","last_became_active":"2024-05-09T07:14:26.591447+0000","last_became_peered":"2024-05-09T07:14:26.591447+0000","last_unstale":"2024-05-09T07:39:49.709609+0000","last_undegraded":"2024-05-09T07:39:49.709609+0000","last_fullsized":"2024-05-09T07:39:49.709609+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2565","last_scrub_stamp":"2024-05-09T07:39:49.709531+0000","last_deep_scrub":"27'2565","last_deep_scrub_stamp":"2024-05-09T07:39:49.709531+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.709531+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:35:00.704431+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"27'7877","reported_seq":7924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.759260+0000","last_change":"2024-05-09T07:39:45.759396+0000","last_active":"2024-05-09T07:39:45.759260+0000","last_peered":"2024-05-09T07:39:45.759260+0000","last_clean":"2024-05-09T07:39:45.759260+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:39:45.759260+0000","last_undegraded":"2024-05-09T07:39:45.759260+0000","last_fullsized":"2024-05-09T07:39:45.759260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7877","last_scrub_stamp":"2024-05-09T07:39:45.759163+0000","last_deep_scrub":"27'7877","last_deep_scrub_stamp":"2024-05-09T07:39:45.759163+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.759163+0000","objects_scrubbed":2,"log_size":7877,"log_dups_size":0,"ondisk_log_size":7877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:34.122563+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7879,"num_write_kb":58494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9733","reported_seq":10223,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.820716+0000","last_change":"2024-05-09T07:39:56.820745+0000","last_active":"2024-05-09T07:39:56.820716+0000","last_peered":"2024-05-09T07:39:56.820716+0000","last_clean":"2024-05-09T07:39:56.820716+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:39:56.820716+0000","last_undegraded":"2024-05-09T07:39:56.820716+0000","last_fullsized":"2024-05-09T07:39:56.820716+0000","mapping_epoch":22,"log_start":"27'1834","ondisk_log_start":"27'1834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9733","last_scrub_stamp":"2024-05-09T07:39:56.820610+0000","last_deep_scrub":"27'9733","last_deep_scrub_stamp":"2024-05-09T07:39:56.820610+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.820610+0000","objects_scrubbed":0,"log_size":7899,"log_dups_size":0,"ondisk_log_size":7899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:22.442335+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":449,"num_read_kb":114046,"num_write":7019,"num_write_kb":962272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2589","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.022806+0000","last_change":"2024-05-09T07:39:51.022834+0000","last_active":"2024-05-09T07:39:51.022806+0000","last_peered":"2024-05-09T07:39:51.022806+0000","last_clean":"2024-05-09T07:39:51.022806+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:39:51.022806+0000","last_undegraded":"2024-05-09T07:39:51.022806+0000","last_fullsized":"2024-05-09T07:39:51.022806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2589","last_scrub_stamp":"2024-05-09T07:39:51.022717+0000","last_deep_scrub":"27'2589","last_deep_scrub_stamp":"2024-05-09T07:39:51.022717+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.022717+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:09.129494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7547","reported_seq":7595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.351048+0000","last_change":"2024-05-09T07:39:45.351076+0000","last_active":"2024-05-09T07:39:45.351048+0000","last_peered":"2024-05-09T07:39:45.351048+0000","last_clean":"2024-05-09T07:39:45.351048+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:39:45.351048+0000","last_undegraded":"2024-05-09T07:39:45.351048+0000","last_fullsized":"2024-05-09T07:39:45.351048+0000","mapping_epoch":17,"log_start":"27'200","ondisk_log_start":"27'200","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7547","last_scrub_stamp":"2024-05-09T07:39:45.350982+0000","last_deep_scrub":"27'7547","last_deep_scrub_stamp":"2024-05-09T07:39:45.350982+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.350982+0000","objects_scrubbed":3,"log_size":7347,"log_dups_size":0,"ondisk_log_size":7347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:45.621741+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7546,"num_write_kb":58281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9829","reported_seq":10121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.624925+0000","last_change":"2024-05-09T07:39:54.624950+0000","last_active":"2024-05-09T07:39:54.624925+0000","last_peered":"2024-05-09T07:39:54.624925+0000","last_clean":"2024-05-09T07:39:54.624925+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:39:54.624925+0000","last_undegraded":"2024-05-09T07:39:54.624925+0000","last_fullsized":"2024-05-09T07:39:54.624925+0000","mapping_epoch":22,"log_start":"27'2499","ondisk_log_start":"27'2499","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9829","last_scrub_stamp":"2024-05-09T07:39:54.624831+0000","last_deep_scrub":"27'9829","last_deep_scrub_stamp":"2024-05-09T07:39:54.624831+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.624831+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:17.835798+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":251,"num_read_kb":63754,"num_write":7056,"num_write_kb":893799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2456","reported_seq":2501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:59.252073+0000","last_change":"2024-05-09T07:39:59.252117+0000","last_active":"2024-05-09T07:39:59.252073+0000","last_peered":"2024-05-09T07:39:59.252073+0000","last_clean":"2024-05-09T07:39:59.252073+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:39:59.252073+0000","last_undegraded":"2024-05-09T07:39:59.252073+0000","last_fullsized":"2024-05-09T07:39:59.252073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2456","last_scrub_stamp":"2024-05-09T07:39:59.251965+0000","last_deep_scrub":"27'2456","last_deep_scrub_stamp":"2024-05-09T07:39:59.251965+0000","last_clean_scrub_stamp":"2024-05-09T07:39:59.251965+0000","objects_scrubbed":0,"log_size":2456,"log_dups_size":0,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:02.422533+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8870","reported_seq":8917,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.871046+0000","last_change":"2024-05-09T07:39:43.871134+0000","last_active":"2024-05-09T07:39:43.871046+0000","last_peered":"2024-05-09T07:39:43.871046+0000","last_clean":"2024-05-09T07:39:43.871046+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:39:43.871046+0000","last_undegraded":"2024-05-09T07:39:43.871046+0000","last_fullsized":"2024-05-09T07:39:43.871046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8870","last_scrub_stamp":"2024-05-09T07:39:43.870938+0000","last_deep_scrub":"27'8870","last_deep_scrub_stamp":"2024-05-09T07:39:43.870938+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.870938+0000","objects_scrubbed":7,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:08.222552+0000","scrub_duration":91,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8854,"num_write_kb":79483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9365","reported_seq":10382,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.197478+0000","last_change":"2024-05-09T07:13:49.941284+0000","last_active":"2024-05-09T07:39:43.197478+0000","last_peered":"2024-05-09T07:39:43.197478+0000","last_clean":"2024-05-09T07:39:43.197478+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:39:43.197478+0000","last_undegraded":"2024-05-09T07:39:43.197478+0000","last_fullsized":"2024-05-09T07:39:43.197478+0000","mapping_epoch":22,"log_start":"27'1982","ondisk_log_start":"27'1982","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":983,"num_read_kb":249682,"num_write":6568,"num_write_kb":1037240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2557","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.275849+0000","last_change":"2024-05-09T07:39:53.275875+0000","last_active":"2024-05-09T07:39:53.275849+0000","last_peered":"2024-05-09T07:39:53.275849+0000","last_clean":"2024-05-09T07:39:53.275849+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:39:53.275849+0000","last_undegraded":"2024-05-09T07:39:53.275849+0000","last_fullsized":"2024-05-09T07:39:53.275849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2557","last_scrub_stamp":"2024-05-09T07:39:53.275754+0000","last_deep_scrub":"27'2557","last_deep_scrub_stamp":"2024-05-09T07:39:53.275754+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.275754+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:39.108105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5467","reported_seq":5514,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.298874+0000","last_change":"2024-05-09T07:39:43.298937+0000","last_active":"2024-05-09T07:39:43.298874+0000","last_peered":"2024-05-09T07:39:43.298874+0000","last_clean":"2024-05-09T07:39:43.298874+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:39:43.298874+0000","last_undegraded":"2024-05-09T07:39:43.298874+0000","last_fullsized":"2024-05-09T07:39:43.298874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5467","last_scrub_stamp":"2024-05-09T07:39:43.298807+0000","last_deep_scrub":"27'5467","last_deep_scrub_stamp":"2024-05-09T07:39:43.298807+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.298807+0000","objects_scrubbed":2,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:09.732787+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5458,"num_write_kb":44489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9305","reported_seq":9873,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.305528+0000","last_change":"2024-05-09T07:39:51.305576+0000","last_active":"2024-05-09T07:39:51.305528+0000","last_peered":"2024-05-09T07:39:51.305528+0000","last_clean":"2024-05-09T07:39:51.305528+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:39:51.305528+0000","last_undegraded":"2024-05-09T07:39:51.305528+0000","last_fullsized":"2024-05-09T07:39:51.305528+0000","mapping_epoch":22,"log_start":"27'2103","ondisk_log_start":"27'2103","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9305","last_scrub_stamp":"2024-05-09T07:39:51.305425+0000","last_deep_scrub":"27'9305","last_deep_scrub_stamp":"2024-05-09T07:39:51.305425+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.305425+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:09.412202+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":527,"num_read_kb":133858,"num_write":6613,"num_write_kb":868619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2624","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:58.263568+0000","last_change":"2024-05-09T07:39:58.263613+0000","last_active":"2024-05-09T07:39:58.263568+0000","last_peered":"2024-05-09T07:39:58.263568+0000","last_clean":"2024-05-09T07:39:58.263568+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:39:58.263568+0000","last_undegraded":"2024-05-09T07:39:58.263568+0000","last_fullsized":"2024-05-09T07:39:58.263568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2624","last_scrub_stamp":"2024-05-09T07:39:58.263467+0000","last_deep_scrub":"27'2624","last_deep_scrub_stamp":"2024-05-09T07:39:58.263467+0000","last_clean_scrub_stamp":"2024-05-09T07:39:58.263467+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:34.588812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3474","reported_seq":3521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.779308+0000","last_change":"2024-05-09T07:39:42.779351+0000","last_active":"2024-05-09T07:39:42.779308+0000","last_peered":"2024-05-09T07:39:42.779308+0000","last_clean":"2024-05-09T07:39:42.779308+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:39:42.779308+0000","last_undegraded":"2024-05-09T07:39:42.779308+0000","last_fullsized":"2024-05-09T07:39:42.779308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3474","last_scrub_stamp":"2024-05-09T07:39:42.779246+0000","last_deep_scrub":"27'3474","last_deep_scrub_stamp":"2024-05-09T07:39:42.779246+0000","last_clean_scrub_stamp":"2024-05-09T07:39:42.779246+0000","objects_scrubbed":6,"log_size":3474,"log_dups_size":0,"ondisk_log_size":3474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:47.753557+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3482,"num_write_kb":39113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9477","reported_seq":10061,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.603567+0000","last_change":"2024-05-09T07:39:53.603611+0000","last_active":"2024-05-09T07:39:53.603567+0000","last_peered":"2024-05-09T07:39:53.603567+0000","last_clean":"2024-05-09T07:39:53.603567+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:39:53.603567+0000","last_undegraded":"2024-05-09T07:39:53.603567+0000","last_fullsized":"2024-05-09T07:39:53.603567+0000","mapping_epoch":22,"log_start":"27'2114","ondisk_log_start":"27'2114","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9477","last_scrub_stamp":"2024-05-09T07:39:53.603458+0000","last_deep_scrub":"27'9477","last_deep_scrub_stamp":"2024-05-09T07:39:53.603458+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.603458+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:30.220063+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":138176,"num_write":6755,"num_write_kb":1181456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2498","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.853727+0000","last_change":"2024-05-09T07:39:48.853781+0000","last_active":"2024-05-09T07:39:48.853727+0000","last_peered":"2024-05-09T07:39:48.853727+0000","last_clean":"2024-05-09T07:39:48.853727+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:39:48.853727+0000","last_undegraded":"2024-05-09T07:39:48.853727+0000","last_fullsized":"2024-05-09T07:39:48.853727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2498","last_scrub_stamp":"2024-05-09T07:39:48.853626+0000","last_deep_scrub":"27'2498","last_deep_scrub_stamp":"2024-05-09T07:39:48.853626+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.853626+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:49.124385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4971","reported_seq":5019,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.976174+0000","last_change":"2024-05-09T07:39:44.976243+0000","last_active":"2024-05-09T07:39:44.976174+0000","last_peered":"2024-05-09T07:39:44.976174+0000","last_clean":"2024-05-09T07:39:44.976174+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:39:44.976174+0000","last_undegraded":"2024-05-09T07:39:44.976174+0000","last_fullsized":"2024-05-09T07:39:44.976174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4971","last_scrub_stamp":"2024-05-09T07:39:44.976095+0000","last_deep_scrub":"27'4971","last_deep_scrub_stamp":"2024-05-09T07:39:44.976095+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.976095+0000","objects_scrubbed":3,"log_size":4971,"log_dups_size":0,"ondisk_log_size":4971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:17.477682+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":4979,"num_write_kb":44143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9318","reported_seq":9622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.681844+0000","last_change":"2024-05-09T07:13:49.941945+0000","last_active":"2024-05-09T07:39:34.681844+0000","last_peered":"2024-05-09T07:39:34.681844+0000","last_clean":"2024-05-09T07:39:34.681844+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:39:34.681844+0000","last_undegraded":"2024-05-09T07:39:34.681844+0000","last_fullsized":"2024-05-09T07:39:34.681844+0000","mapping_epoch":22,"log_start":"27'3038","ondisk_log_start":"27'3038","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":69088,"num_write":6517,"num_write_kb":924175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2603","reported_seq":2648,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:00.196618+0000","last_change":"2024-05-09T07:40:00.196646+0000","last_active":"2024-05-09T07:40:00.196618+0000","last_peered":"2024-05-09T07:40:00.196618+0000","last_clean":"2024-05-09T07:40:00.196618+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:40:00.196618+0000","last_undegraded":"2024-05-09T07:40:00.196618+0000","last_fullsized":"2024-05-09T07:40:00.196618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2603","last_scrub_stamp":"2024-05-09T07:40:00.196534+0000","last_deep_scrub":"27'2603","last_deep_scrub_stamp":"2024-05-09T07:40:00.196534+0000","last_clean_scrub_stamp":"2024-05-09T07:40:00.196534+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:31.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2653","reported_seq":2701,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.119973+0000","last_change":"2024-05-09T07:39:46.120005+0000","last_active":"2024-05-09T07:39:46.119973+0000","last_peered":"2024-05-09T07:39:46.119973+0000","last_clean":"2024-05-09T07:39:46.119973+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:39:46.119973+0000","last_undegraded":"2024-05-09T07:39:46.119973+0000","last_fullsized":"2024-05-09T07:39:46.119973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2653","last_scrub_stamp":"2024-05-09T07:39:46.119886+0000","last_deep_scrub":"27'2653","last_deep_scrub_stamp":"2024-05-09T07:39:46.119886+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.119886+0000","objects_scrubbed":2,"log_size":2653,"log_dups_size":0,"ondisk_log_size":2653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.572755+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2652,"num_write_kb":30266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9224","reported_seq":9454,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.287892+0000","last_change":"2024-05-09T07:13:49.939563+0000","last_active":"2024-05-09T07:39:42.287892+0000","last_peered":"2024-05-09T07:39:42.287892+0000","last_clean":"2024-05-09T07:39:42.287892+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:39:42.287892+0000","last_undegraded":"2024-05-09T07:39:42.287892+0000","last_fullsized":"2024-05-09T07:39:42.287892+0000","mapping_epoch":22,"log_start":"27'2939","ondisk_log_start":"27'2939","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6285,"log_dups_size":0,"ondisk_log_size":6285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":197,"num_read_kb":50038,"num_write":6664,"num_write_kb":910804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2562","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.559321+0000","last_change":"2024-05-09T07:39:51.559344+0000","last_active":"2024-05-09T07:39:51.559321+0000","last_peered":"2024-05-09T07:39:51.559321+0000","last_clean":"2024-05-09T07:39:51.559321+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:39:51.559321+0000","last_undegraded":"2024-05-09T07:39:51.559321+0000","last_fullsized":"2024-05-09T07:39:51.559321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2562","last_scrub_stamp":"2024-05-09T07:39:51.559252+0000","last_deep_scrub":"27'2562","last_deep_scrub_stamp":"2024-05-09T07:39:51.559252+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.559252+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:07.899467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'7004","reported_seq":7051,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.717054+0000","last_change":"2024-05-09T07:39:42.717102+0000","last_active":"2024-05-09T07:39:42.717054+0000","last_peered":"2024-05-09T07:39:42.717054+0000","last_clean":"2024-05-09T07:39:42.717054+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:39:42.717054+0000","last_undegraded":"2024-05-09T07:39:42.717054+0000","last_fullsized":"2024-05-09T07:39:42.717054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7004","last_scrub_stamp":"2024-05-09T07:39:42.716991+0000","last_deep_scrub":"27'7004","last_deep_scrub_stamp":"2024-05-09T07:39:42.716991+0000","last_clean_scrub_stamp":"2024-05-09T07:39:42.716991+0000","objects_scrubbed":7,"log_size":7004,"log_dups_size":0,"ondisk_log_size":7004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:48.849006+0000","scrub_duration":56,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6989,"num_write_kb":57820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9732","reported_seq":10001,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.228146+0000","last_change":"2024-05-09T07:13:49.943391+0000","last_active":"2024-05-09T07:39:37.228146+0000","last_peered":"2024-05-09T07:39:37.228146+0000","last_clean":"2024-05-09T07:39:37.228146+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:39:37.228146+0000","last_undegraded":"2024-05-09T07:39:37.228146+0000","last_fullsized":"2024-05-09T07:39:37.228146+0000","mapping_epoch":22,"log_start":"27'2359","ondisk_log_start":"27'2359","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7373,"log_dups_size":0,"ondisk_log_size":7373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":60198,"num_write":7045,"num_write_kb":1050276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2506","reported_seq":2550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.427897+0000","last_change":"2024-05-09T07:39:44.427929+0000","last_active":"2024-05-09T07:39:44.427897+0000","last_peered":"2024-05-09T07:39:44.427897+0000","last_clean":"2024-05-09T07:39:44.427897+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:39:44.427897+0000","last_undegraded":"2024-05-09T07:39:44.427897+0000","last_fullsized":"2024-05-09T07:39:44.427897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2506","last_scrub_stamp":"2024-05-09T07:39:44.427778+0000","last_deep_scrub":"27'2506","last_deep_scrub_stamp":"2024-05-09T07:39:44.427778+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.427778+0000","objects_scrubbed":0,"log_size":2506,"log_dups_size":0,"ondisk_log_size":2506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:55.077769+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10547","reported_seq":11222,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.946840+0000","last_change":"2024-05-09T07:39:45.946881+0000","last_active":"2024-05-09T07:39:45.946840+0000","last_peered":"2024-05-09T07:39:45.946840+0000","last_clean":"2024-05-09T07:39:45.946840+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:39:45.946840+0000","last_undegraded":"2024-05-09T07:39:45.946840+0000","last_fullsized":"2024-05-09T07:39:45.946840+0000","mapping_epoch":17,"log_start":"27'2600","ondisk_log_start":"27'2600","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10547","last_scrub_stamp":"2024-05-09T07:39:45.946752+0000","last_deep_scrub":"27'10547","last_deep_scrub_stamp":"2024-05-09T07:39:45.946752+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.946752+0000","objects_scrubbed":5,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:56.596743+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":627,"num_read_kb":4520,"num_write":10536,"num_write_kb":76795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9196","reported_seq":9440,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:37.211239+0000","last_change":"2024-05-09T07:14:26.592176+0000","last_active":"2024-05-09T07:39:37.211239+0000","last_peered":"2024-05-09T07:39:37.211239+0000","last_clean":"2024-05-09T07:39:37.211239+0000","last_became_active":"2024-05-09T07:14:26.590721+0000","last_became_peered":"2024-05-09T07:14:26.590721+0000","last_unstale":"2024-05-09T07:39:37.211239+0000","last_undegraded":"2024-05-09T07:39:37.211239+0000","last_fullsized":"2024-05-09T07:39:37.211239+0000","mapping_epoch":26,"log_start":"27'1828","ondisk_log_start":"27'1828","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":56134,"num_write":6509,"num_write_kb":978701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"27'2577","reported_seq":2622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.287304+0000","last_change":"2024-05-09T07:39:47.287327+0000","last_active":"2024-05-09T07:39:47.287304+0000","last_peered":"2024-05-09T07:39:47.287304+0000","last_clean":"2024-05-09T07:39:47.287304+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:39:47.287304+0000","last_undegraded":"2024-05-09T07:39:47.287304+0000","last_fullsized":"2024-05-09T07:39:47.287304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2577","last_scrub_stamp":"2024-05-09T07:39:47.287227+0000","last_deep_scrub":"27'2577","last_deep_scrub_stamp":"2024-05-09T07:39:47.287227+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.287227+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:46.335272+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6180","reported_seq":6227,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.313502+0000","last_change":"2024-05-09T07:39:46.313543+0000","last_active":"2024-05-09T07:39:46.313502+0000","last_peered":"2024-05-09T07:39:46.313502+0000","last_clean":"2024-05-09T07:39:46.313502+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:39:46.313502+0000","last_undegraded":"2024-05-09T07:39:46.313502+0000","last_fullsized":"2024-05-09T07:39:46.313502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6180","last_scrub_stamp":"2024-05-09T07:39:46.313400+0000","last_deep_scrub":"27'6180","last_deep_scrub_stamp":"2024-05-09T07:39:46.313400+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.313400+0000","objects_scrubbed":2,"log_size":6180,"log_dups_size":0,"ondisk_log_size":6180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:45.361444+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6185,"num_write_kb":41396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9171","reported_seq":9524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.250909+0000","last_change":"2024-05-09T07:39:57.250938+0000","last_active":"2024-05-09T07:39:57.250909+0000","last_peered":"2024-05-09T07:39:57.250909+0000","last_clean":"2024-05-09T07:39:57.250909+0000","last_became_active":"2024-05-09T07:14:26.595936+0000","last_became_peered":"2024-05-09T07:14:26.595936+0000","last_unstale":"2024-05-09T07:39:57.250909+0000","last_undegraded":"2024-05-09T07:39:57.250909+0000","last_fullsized":"2024-05-09T07:39:57.250909+0000","mapping_epoch":26,"log_start":"27'2010","ondisk_log_start":"27'2010","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9171","last_scrub_stamp":"2024-05-09T07:39:57.250803+0000","last_deep_scrub":"27'9171","last_deep_scrub_stamp":"2024-05-09T07:39:57.250803+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.250803+0000","objects_scrubbed":0,"log_size":7161,"log_dups_size":0,"ondisk_log_size":7161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:53.493110+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":305,"num_read_kb":77470,"num_write":6446,"num_write_kb":989255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2611","reported_seq":2655,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:41.667993+0000","last_change":"2024-05-09T07:39:41.668036+0000","last_active":"2024-05-09T07:39:41.667993+0000","last_peered":"2024-05-09T07:39:41.667993+0000","last_clean":"2024-05-09T07:39:41.667993+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:39:41.667993+0000","last_undegraded":"2024-05-09T07:39:41.667993+0000","last_fullsized":"2024-05-09T07:39:41.667993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2611","last_scrub_stamp":"2024-05-09T07:39:41.667866+0000","last_deep_scrub":"27'2611","last_deep_scrub_stamp":"2024-05-09T07:39:41.667866+0000","last_clean_scrub_stamp":"2024-05-09T07:39:41.667866+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:52.317856+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5743","reported_seq":5790,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.085294+0000","last_change":"2024-05-09T07:39:50.085331+0000","last_active":"2024-05-09T07:39:50.085294+0000","last_peered":"2024-05-09T07:39:50.085294+0000","last_clean":"2024-05-09T07:39:50.085294+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:39:50.085294+0000","last_undegraded":"2024-05-09T07:39:50.085294+0000","last_fullsized":"2024-05-09T07:39:50.085294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5743","last_scrub_stamp":"2024-05-09T07:39:50.085191+0000","last_deep_scrub":"27'5743","last_deep_scrub_stamp":"2024-05-09T07:39:50.085191+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.085191+0000","objects_scrubbed":2,"log_size":5743,"log_dups_size":0,"ondisk_log_size":5743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:40.947993+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5742,"num_write_kb":52939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9121","reported_seq":9497,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.329311+0000","last_change":"2024-05-09T07:39:57.329372+0000","last_active":"2024-05-09T07:39:57.329311+0000","last_peered":"2024-05-09T07:39:57.329311+0000","last_clean":"2024-05-09T07:39:57.329311+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:39:57.329311+0000","last_undegraded":"2024-05-09T07:39:57.329311+0000","last_fullsized":"2024-05-09T07:39:57.329311+0000","mapping_epoch":22,"log_start":"27'2805","ondisk_log_start":"27'2805","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9121","last_scrub_stamp":"2024-05-09T07:39:57.329183+0000","last_deep_scrub":"27'9121","last_deep_scrub_stamp":"2024-05-09T07:39:57.329183+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.329183+0000","objects_scrubbed":0,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:56.341286+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":335,"num_read_kb":85090,"num_write":6449,"num_write_kb":967440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2587","reported_seq":2632,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.985851+0000","last_change":"2024-05-09T07:39:54.985876+0000","last_active":"2024-05-09T07:39:54.985851+0000","last_peered":"2024-05-09T07:39:54.985851+0000","last_clean":"2024-05-09T07:39:54.985851+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:39:54.985851+0000","last_undegraded":"2024-05-09T07:39:54.985851+0000","last_fullsized":"2024-05-09T07:39:54.985851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2587","last_scrub_stamp":"2024-05-09T07:39:54.985767+0000","last_deep_scrub":"27'2587","last_deep_scrub_stamp":"2024-05-09T07:39:54.985767+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.985767+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:31.602372+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'5946","reported_seq":6859,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.892149+0000","last_change":"2024-05-09T07:39:46.892175+0000","last_active":"2024-05-09T07:39:46.892149+0000","last_peered":"2024-05-09T07:39:46.892149+0000","last_clean":"2024-05-09T07:39:46.892149+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:39:46.892149+0000","last_undegraded":"2024-05-09T07:39:46.892149+0000","last_fullsized":"2024-05-09T07:39:46.892149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5946","last_scrub_stamp":"2024-05-09T07:39:46.892064+0000","last_deep_scrub":"27'5946","last_deep_scrub_stamp":"2024-05-09T07:39:46.892064+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.892064+0000","objects_scrubbed":0,"log_size":5946,"log_dups_size":0,"ondisk_log_size":5946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:00.006440+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":867,"num_read_kb":4659,"num_write":5947,"num_write_kb":49048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9603","reported_seq":9992,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:34.665090+0000","last_change":"2024-05-09T07:13:49.940958+0000","last_active":"2024-05-09T07:39:34.665090+0000","last_peered":"2024-05-09T07:39:34.665090+0000","last_clean":"2024-05-09T07:39:34.665090+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:39:34.665090+0000","last_undegraded":"2024-05-09T07:39:34.665090+0000","last_fullsized":"2024-05-09T07:39:34.665090+0000","mapping_epoch":22,"log_start":"27'3323","ondisk_log_start":"27'3323","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T07:13:48.478581+0000","last_clean_scrub_stamp":"2024-05-09T07:13:48.478581+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6822,"num_write_kb":1059940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9164","reported_seq":9401,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.557104+0000","last_change":"2024-05-09T07:39:57.557104+0000","last_active":"2024-05-09T07:39:57.557104+0000","last_peered":"2024-05-09T07:39:57.557104+0000","last_clean":"2024-05-09T07:39:57.557104+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:39:57.557104+0000","last_undegraded":"2024-05-09T07:39:57.557104+0000","last_fullsized":"2024-05-09T07:39:57.557104+0000","mapping_epoch":22,"log_start":"27'1813","ondisk_log_start":"27'1813","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9164","last_scrub_stamp":"2024-05-09T07:39:57.556982+0000","last_deep_scrub":"27'9164","last_deep_scrub_stamp":"2024-05-09T07:39:57.556982+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.556982+0000","objects_scrubbed":0,"log_size":7351,"log_dups_size":0,"ondisk_log_size":7351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:28.408443+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":49276,"num_write":6561,"num_write_kb":912358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2535","reported_seq":2580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.380110+0000","last_change":"2024-05-09T07:39:51.380151+0000","last_active":"2024-05-09T07:39:51.380110+0000","last_peered":"2024-05-09T07:39:51.380110+0000","last_clean":"2024-05-09T07:39:51.380110+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:39:51.380110+0000","last_undegraded":"2024-05-09T07:39:51.380110+0000","last_fullsized":"2024-05-09T07:39:51.380110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2535","last_scrub_stamp":"2024-05-09T07:39:51.380024+0000","last_deep_scrub":"27'2535","last_deep_scrub_stamp":"2024-05-09T07:39:51.380024+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.380024+0000","objects_scrubbed":0,"log_size":2535,"log_dups_size":0,"ondisk_log_size":2535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:43:51.708529+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9072","reported_seq":9119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.101955+0000","last_change":"2024-05-09T07:39:47.101986+0000","last_active":"2024-05-09T07:39:47.101955+0000","last_peered":"2024-05-09T07:39:47.101955+0000","last_clean":"2024-05-09T07:39:47.101955+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:39:47.101955+0000","last_undegraded":"2024-05-09T07:39:47.101955+0000","last_fullsized":"2024-05-09T07:39:47.101955+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9072","last_scrub_stamp":"2024-05-09T07:39:47.101861+0000","last_deep_scrub":"27'9072","last_deep_scrub_stamp":"2024-05-09T07:39:47.101861+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.101861+0000","objects_scrubbed":1,"log_size":7372,"log_dups_size":0,"ondisk_log_size":7372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:22.953252+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9068,"num_write_kb":67566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8844","reported_seq":10039,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.768345+0000","last_change":"2024-05-09T07:39:55.768399+0000","last_active":"2024-05-09T07:39:55.768345+0000","last_peered":"2024-05-09T07:39:55.768345+0000","last_clean":"2024-05-09T07:39:55.768345+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:39:55.768345+0000","last_undegraded":"2024-05-09T07:39:55.768345+0000","last_fullsized":"2024-05-09T07:39:55.768345+0000","mapping_epoch":22,"log_start":"27'900","ondisk_log_start":"27'900","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8844","last_scrub_stamp":"2024-05-09T07:39:55.768221+0000","last_deep_scrub":"27'8844","last_deep_scrub_stamp":"2024-05-09T07:39:55.768221+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.768221+0000","objects_scrubbed":0,"log_size":7944,"log_dups_size":0,"ondisk_log_size":7944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:12.108436+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1153,"num_read_kb":292862,"num_write":6082,"num_write_kb":877058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2633","reported_seq":2678,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.253693+0000","last_change":"2024-05-09T07:39:52.253732+0000","last_active":"2024-05-09T07:39:52.253693+0000","last_peered":"2024-05-09T07:39:52.253693+0000","last_clean":"2024-05-09T07:39:52.253693+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:39:52.253693+0000","last_undegraded":"2024-05-09T07:39:52.253693+0000","last_fullsized":"2024-05-09T07:39:52.253693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2633","last_scrub_stamp":"2024-05-09T07:39:52.253615+0000","last_deep_scrub":"27'2633","last_deep_scrub_stamp":"2024-05-09T07:39:52.253615+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.253615+0000","objects_scrubbed":0,"log_size":2633,"log_dups_size":0,"ondisk_log_size":2633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:03.018472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5358","reported_seq":5405,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.297604+0000","last_change":"2024-05-09T07:39:50.297633+0000","last_active":"2024-05-09T07:39:50.297604+0000","last_peered":"2024-05-09T07:39:50.297604+0000","last_clean":"2024-05-09T07:39:50.297604+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:39:50.297604+0000","last_undegraded":"2024-05-09T07:39:50.297604+0000","last_fullsized":"2024-05-09T07:39:50.297604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5358","last_scrub_stamp":"2024-05-09T07:39:50.297534+0000","last_deep_scrub":"27'5358","last_deep_scrub_stamp":"2024-05-09T07:39:50.297534+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.297534+0000","objects_scrubbed":2,"log_size":5358,"log_dups_size":0,"ondisk_log_size":5358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:06.637749+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5356,"num_write_kb":44481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9698","reported_seq":9745,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.997220+0000","last_change":"2024-05-09T07:39:43.997249+0000","last_active":"2024-05-09T07:39:43.997220+0000","last_peered":"2024-05-09T07:39:43.997220+0000","last_clean":"2024-05-09T07:39:43.997220+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:39:43.997220+0000","last_undegraded":"2024-05-09T07:39:43.997220+0000","last_fullsized":"2024-05-09T07:39:43.997220+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9698","last_scrub_stamp":"2024-05-09T07:39:43.997141+0000","last_deep_scrub":"27'9698","last_deep_scrub_stamp":"2024-05-09T07:39:43.997141+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.997141+0000","objects_scrubbed":1,"log_size":7898,"log_dups_size":0,"ondisk_log_size":7898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:24.963666+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9694,"num_write_kb":72019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2561","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.709885+0000","last_change":"2024-05-09T07:39:46.709907+0000","last_active":"2024-05-09T07:39:46.709885+0000","last_peered":"2024-05-09T07:39:46.709885+0000","last_clean":"2024-05-09T07:39:46.709885+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:39:46.709885+0000","last_undegraded":"2024-05-09T07:39:46.709885+0000","last_fullsized":"2024-05-09T07:39:46.709885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2561","last_scrub_stamp":"2024-05-09T07:39:46.709806+0000","last_deep_scrub":"27'2561","last_deep_scrub_stamp":"2024-05-09T07:39:46.709806+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.709806+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:45.055437+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9401","reported_seq":9465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.703699+0000","last_change":"2024-05-09T07:39:48.703725+0000","last_active":"2024-05-09T07:39:48.703699+0000","last_peered":"2024-05-09T07:39:48.703699+0000","last_clean":"2024-05-09T07:39:48.703699+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:39:48.703699+0000","last_undegraded":"2024-05-09T07:39:48.703699+0000","last_fullsized":"2024-05-09T07:39:48.703699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9401","last_scrub_stamp":"2024-05-09T07:39:48.703611+0000","last_deep_scrub":"27'9401","last_deep_scrub_stamp":"2024-05-09T07:39:48.703611+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.703611+0000","objects_scrubbed":0,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:29.670136+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":6096,"num_write":6603,"num_write_kb":983207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8029","reported_seq":8076,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.228151+0000","last_change":"2024-05-09T07:39:49.228172+0000","last_active":"2024-05-09T07:39:49.228151+0000","last_peered":"2024-05-09T07:39:49.228151+0000","last_clean":"2024-05-09T07:39:49.228151+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:39:49.228151+0000","last_undegraded":"2024-05-09T07:39:49.228151+0000","last_fullsized":"2024-05-09T07:39:49.228151+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8029","last_scrub_stamp":"2024-05-09T07:39:49.228089+0000","last_deep_scrub":"27'8029","last_deep_scrub_stamp":"2024-05-09T07:39:49.228089+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.228089+0000","objects_scrubbed":0,"log_size":6329,"log_dups_size":0,"ondisk_log_size":6329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:35.060440+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8031,"num_write_kb":66635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2603","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.246682+0000","last_change":"2024-05-09T07:39:45.246722+0000","last_active":"2024-05-09T07:39:45.246682+0000","last_peered":"2024-05-09T07:39:45.246682+0000","last_clean":"2024-05-09T07:39:45.246682+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:39:45.246682+0000","last_undegraded":"2024-05-09T07:39:45.246682+0000","last_fullsized":"2024-05-09T07:39:45.246682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2603","last_scrub_stamp":"2024-05-09T07:39:45.246606+0000","last_deep_scrub":"27'2603","last_deep_scrub_stamp":"2024-05-09T07:39:45.246606+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.246606+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:22.699475+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9830","reported_seq":9991,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:58.386218+0000","last_change":"2024-05-09T07:39:58.386260+0000","last_active":"2024-05-09T07:39:58.386218+0000","last_peered":"2024-05-09T07:39:58.386218+0000","last_clean":"2024-05-09T07:39:58.386218+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:39:58.386218+0000","last_undegraded":"2024-05-09T07:39:58.386218+0000","last_fullsized":"2024-05-09T07:39:58.386218+0000","mapping_epoch":22,"log_start":"27'3543","ondisk_log_start":"27'3543","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9830","last_scrub_stamp":"2024-05-09T07:39:58.386093+0000","last_deep_scrub":"27'9830","last_deep_scrub_stamp":"2024-05-09T07:39:58.386093+0000","last_clean_scrub_stamp":"2024-05-09T07:39:58.386093+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:26:11.093394+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":30734,"num_write":7017,"num_write_kb":1030226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2510","reported_seq":2557,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.807824+0000","last_change":"2024-05-09T07:39:51.807849+0000","last_active":"2024-05-09T07:39:51.807824+0000","last_peered":"2024-05-09T07:39:51.807824+0000","last_clean":"2024-05-09T07:39:51.807824+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:39:51.807824+0000","last_undegraded":"2024-05-09T07:39:51.807824+0000","last_fullsized":"2024-05-09T07:39:51.807824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2510","last_scrub_stamp":"2024-05-09T07:39:51.807731+0000","last_deep_scrub":"27'2510","last_deep_scrub_stamp":"2024-05-09T07:39:51.807731+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.807731+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:56.985027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2625","reported_seq":2671,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.687922+0000","last_change":"2024-05-09T07:39:46.687962+0000","last_active":"2024-05-09T07:39:46.687922+0000","last_peered":"2024-05-09T07:39:46.687922+0000","last_clean":"2024-05-09T07:39:46.687922+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:39:46.687922+0000","last_undegraded":"2024-05-09T07:39:46.687922+0000","last_fullsized":"2024-05-09T07:39:46.687922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2625","last_scrub_stamp":"2024-05-09T07:39:46.687817+0000","last_deep_scrub":"27'2625","last_deep_scrub_stamp":"2024-05-09T07:39:46.687817+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.687817+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:09:28.909565+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2528","reported_seq":2575,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.008243+0000","last_change":"2024-05-09T07:39:54.008268+0000","last_active":"2024-05-09T07:39:54.008243+0000","last_peered":"2024-05-09T07:39:54.008243+0000","last_clean":"2024-05-09T07:39:54.008243+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:39:54.008243+0000","last_undegraded":"2024-05-09T07:39:54.008243+0000","last_fullsized":"2024-05-09T07:39:54.008243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2528","last_scrub_stamp":"2024-05-09T07:39:54.008158+0000","last_deep_scrub":"27'2528","last_deep_scrub_stamp":"2024-05-09T07:39:54.008158+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.008158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:51.314898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2612","reported_seq":2658,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.662490+0000","last_change":"2024-05-09T07:39:47.662514+0000","last_active":"2024-05-09T07:39:47.662490+0000","last_peered":"2024-05-09T07:39:47.662490+0000","last_clean":"2024-05-09T07:39:47.662490+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:39:47.662490+0000","last_undegraded":"2024-05-09T07:39:47.662490+0000","last_fullsized":"2024-05-09T07:39:47.662490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2612","last_scrub_stamp":"2024-05-09T07:39:47.662413+0000","last_deep_scrub":"27'2612","last_deep_scrub_stamp":"2024-05-09T07:39:47.662413+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.662413+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:52.839709+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2606","reported_seq":2653,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.271140+0000","last_change":"2024-05-09T07:39:54.271164+0000","last_active":"2024-05-09T07:39:54.271140+0000","last_peered":"2024-05-09T07:39:54.271140+0000","last_clean":"2024-05-09T07:39:54.271140+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:39:54.271140+0000","last_undegraded":"2024-05-09T07:39:54.271140+0000","last_fullsized":"2024-05-09T07:39:54.271140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2606","last_scrub_stamp":"2024-05-09T07:39:54.271071+0000","last_deep_scrub":"27'2606","last_deep_scrub_stamp":"2024-05-09T07:39:54.271071+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.271071+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:30.596415+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2485","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.352606+0000","last_change":"2024-05-09T07:39:50.352606+0000","last_active":"2024-05-09T07:39:50.352606+0000","last_peered":"2024-05-09T07:39:50.352606+0000","last_clean":"2024-05-09T07:39:50.352606+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:39:50.352606+0000","last_undegraded":"2024-05-09T07:39:50.352606+0000","last_fullsized":"2024-05-09T07:39:50.352606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2485","last_scrub_stamp":"2024-05-09T07:39:50.352482+0000","last_deep_scrub":"27'2485","last_deep_scrub_stamp":"2024-05-09T07:39:50.352482+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.352482+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:16:47.994271+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2622","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.782107+0000","last_change":"2024-05-09T07:39:54.782134+0000","last_active":"2024-05-09T07:39:54.782107+0000","last_peered":"2024-05-09T07:39:54.782107+0000","last_clean":"2024-05-09T07:39:54.782107+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:39:54.782107+0000","last_undegraded":"2024-05-09T07:39:54.782107+0000","last_fullsized":"2024-05-09T07:39:54.782107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2622","last_scrub_stamp":"2024-05-09T07:39:54.782014+0000","last_deep_scrub":"27'2622","last_deep_scrub_stamp":"2024-05-09T07:39:54.782014+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.782014+0000","objects_scrubbed":0,"log_size":2622,"log_dups_size":0,"ondisk_log_size":2622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:40.614366+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2594","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.282537+0000","last_change":"2024-05-09T07:39:55.282577+0000","last_active":"2024-05-09T07:39:55.282537+0000","last_peered":"2024-05-09T07:39:55.282537+0000","last_clean":"2024-05-09T07:39:55.282537+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:39:55.282537+0000","last_undegraded":"2024-05-09T07:39:55.282537+0000","last_fullsized":"2024-05-09T07:39:55.282537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2594","last_scrub_stamp":"2024-05-09T07:39:55.282417+0000","last_deep_scrub":"27'2594","last_deep_scrub_stamp":"2024-05-09T07:39:55.282417+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.282417+0000","objects_scrubbed":0,"log_size":2594,"log_dups_size":0,"ondisk_log_size":2594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.452985+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2576","reported_seq":2606,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.702895+0000","last_change":"2024-05-09T07:39:51.702917+0000","last_active":"2024-05-09T07:39:51.702895+0000","last_peered":"2024-05-09T07:39:51.702895+0000","last_clean":"2024-05-09T07:39:51.702895+0000","last_became_active":"2024-05-09T07:14:26.646058+0000","last_became_peered":"2024-05-09T07:14:26.646058+0000","last_unstale":"2024-05-09T07:39:51.702895+0000","last_undegraded":"2024-05-09T07:39:51.702895+0000","last_fullsized":"2024-05-09T07:39:51.702895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2576","last_scrub_stamp":"2024-05-09T07:39:51.702818+0000","last_deep_scrub":"27'2576","last_deep_scrub_stamp":"2024-05-09T07:39:51.702818+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.702818+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:04.817195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'2541","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.230572+0000","last_change":"2024-05-09T07:39:47.230610+0000","last_active":"2024-05-09T07:39:47.230572+0000","last_peered":"2024-05-09T07:39:47.230572+0000","last_clean":"2024-05-09T07:39:47.230572+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:39:47.230572+0000","last_undegraded":"2024-05-09T07:39:47.230572+0000","last_fullsized":"2024-05-09T07:39:47.230572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2541","last_scrub_stamp":"2024-05-09T07:39:47.230473+0000","last_deep_scrub":"27'2541","last_deep_scrub_stamp":"2024-05-09T07:39:47.230473+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.230473+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:16.973851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2609","reported_seq":2655,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.981280+0000","last_change":"2024-05-09T07:39:56.981303+0000","last_active":"2024-05-09T07:39:56.981280+0000","last_peered":"2024-05-09T07:39:56.981280+0000","last_clean":"2024-05-09T07:39:56.981280+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:39:56.981280+0000","last_undegraded":"2024-05-09T07:39:56.981280+0000","last_fullsized":"2024-05-09T07:39:56.981280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2609","last_scrub_stamp":"2024-05-09T07:39:56.981197+0000","last_deep_scrub":"27'2609","last_deep_scrub_stamp":"2024-05-09T07:39:56.981197+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.981197+0000","objects_scrubbed":0,"log_size":2609,"log_dups_size":0,"ondisk_log_size":2609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:53.223504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2621","reported_seq":2666,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.561840+0000","last_change":"2024-05-09T07:39:52.561884+0000","last_active":"2024-05-09T07:39:52.561840+0000","last_peered":"2024-05-09T07:39:52.561840+0000","last_clean":"2024-05-09T07:39:52.561840+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:39:52.561840+0000","last_undegraded":"2024-05-09T07:39:52.561840+0000","last_fullsized":"2024-05-09T07:39:52.561840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2621","last_scrub_stamp":"2024-05-09T07:39:52.561754+0000","last_deep_scrub":"27'2621","last_deep_scrub_stamp":"2024-05-09T07:39:52.561754+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.561754+0000","objects_scrubbed":0,"log_size":2621,"log_dups_size":0,"ondisk_log_size":2621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:49.868495+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2611","reported_seq":2657,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.824213+0000","last_change":"2024-05-09T07:39:52.824245+0000","last_active":"2024-05-09T07:39:52.824213+0000","last_peered":"2024-05-09T07:39:52.824213+0000","last_clean":"2024-05-09T07:39:52.824213+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:39:52.824213+0000","last_undegraded":"2024-05-09T07:39:52.824213+0000","last_fullsized":"2024-05-09T07:39:52.824213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2611","last_scrub_stamp":"2024-05-09T07:39:52.824140+0000","last_deep_scrub":"27'2611","last_deep_scrub_stamp":"2024-05-09T07:39:52.824140+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.824140+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:22.567518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2615","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.292428+0000","last_change":"2024-05-09T07:39:54.292451+0000","last_active":"2024-05-09T07:39:54.292428+0000","last_peered":"2024-05-09T07:39:54.292428+0000","last_clean":"2024-05-09T07:39:54.292428+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:39:54.292428+0000","last_undegraded":"2024-05-09T07:39:54.292428+0000","last_fullsized":"2024-05-09T07:39:54.292428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2615","last_scrub_stamp":"2024-05-09T07:39:54.292348+0000","last_deep_scrub":"27'2615","last_deep_scrub_stamp":"2024-05-09T07:39:54.292348+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.292348+0000","objects_scrubbed":0,"log_size":2615,"log_dups_size":0,"ondisk_log_size":2615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:10.632563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2554","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.689317+0000","last_change":"2024-05-09T07:39:47.689358+0000","last_active":"2024-05-09T07:39:47.689317+0000","last_peered":"2024-05-09T07:39:47.689317+0000","last_clean":"2024-05-09T07:39:47.689317+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:39:47.689317+0000","last_undegraded":"2024-05-09T07:39:47.689317+0000","last_fullsized":"2024-05-09T07:39:47.689317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2554","last_scrub_stamp":"2024-05-09T07:39:47.689226+0000","last_deep_scrub":"27'2554","last_deep_scrub_stamp":"2024-05-09T07:39:47.689226+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.689226+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:28.357856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2625","reported_seq":2671,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:58.017424+0000","last_change":"2024-05-09T07:39:58.017448+0000","last_active":"2024-05-09T07:39:58.017424+0000","last_peered":"2024-05-09T07:39:58.017424+0000","last_clean":"2024-05-09T07:39:58.017424+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:39:58.017424+0000","last_undegraded":"2024-05-09T07:39:58.017424+0000","last_fullsized":"2024-05-09T07:39:58.017424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2625","last_scrub_stamp":"2024-05-09T07:39:58.017310+0000","last_deep_scrub":"27'2625","last_deep_scrub_stamp":"2024-05-09T07:39:58.017310+0000","last_clean_scrub_stamp":"2024-05-09T07:39:58.017310+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:01.187878+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2558","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.092661+0000","last_change":"2024-05-09T07:39:44.092687+0000","last_active":"2024-05-09T07:39:44.092661+0000","last_peered":"2024-05-09T07:39:44.092661+0000","last_clean":"2024-05-09T07:39:44.092661+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:39:44.092661+0000","last_undegraded":"2024-05-09T07:39:44.092661+0000","last_fullsized":"2024-05-09T07:39:44.092661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-05-09T07:39:44.092580+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-05-09T07:39:44.092580+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.092580+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:10.526561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2678","reported_seq":2724,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.349461+0000","last_change":"2024-05-09T07:39:48.349500+0000","last_active":"2024-05-09T07:39:48.349461+0000","last_peered":"2024-05-09T07:39:48.349461+0000","last_clean":"2024-05-09T07:39:48.349461+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:39:48.349461+0000","last_undegraded":"2024-05-09T07:39:48.349461+0000","last_fullsized":"2024-05-09T07:39:48.349461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2678","last_scrub_stamp":"2024-05-09T07:39:48.349373+0000","last_deep_scrub":"27'2678","last_deep_scrub_stamp":"2024-05-09T07:39:48.349373+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.349373+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:29.340354+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2529","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.630361+0000","last_change":"2024-05-09T07:39:48.630402+0000","last_active":"2024-05-09T07:39:48.630361+0000","last_peered":"2024-05-09T07:39:48.630361+0000","last_clean":"2024-05-09T07:39:48.630361+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:39:48.630361+0000","last_undegraded":"2024-05-09T07:39:48.630361+0000","last_fullsized":"2024-05-09T07:39:48.630361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-05-09T07:39:48.630274+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-05-09T07:39:48.630274+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.630274+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:39.493076+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2521","reported_seq":2566,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.217135+0000","last_change":"2024-05-09T07:39:44.217179+0000","last_active":"2024-05-09T07:39:44.217135+0000","last_peered":"2024-05-09T07:39:44.217135+0000","last_clean":"2024-05-09T07:39:44.217135+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:39:44.217135+0000","last_undegraded":"2024-05-09T07:39:44.217135+0000","last_fullsized":"2024-05-09T07:39:44.217135+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2521","last_scrub_stamp":"2024-05-09T07:39:44.216997+0000","last_deep_scrub":"27'2521","last_deep_scrub_stamp":"2024-05-09T07:39:44.216997+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.216997+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:43.265042+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2490","reported_seq":2534,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.335357+0000","last_change":"2024-05-09T07:39:48.335382+0000","last_active":"2024-05-09T07:39:48.335357+0000","last_peered":"2024-05-09T07:39:48.335357+0000","last_clean":"2024-05-09T07:39:48.335357+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:39:48.335357+0000","last_undegraded":"2024-05-09T07:39:48.335357+0000","last_fullsized":"2024-05-09T07:39:48.335357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2490","last_scrub_stamp":"2024-05-09T07:39:48.335273+0000","last_deep_scrub":"27'2490","last_deep_scrub_stamp":"2024-05-09T07:39:48.335273+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.335273+0000","objects_scrubbed":0,"log_size":2490,"log_dups_size":0,"ondisk_log_size":2490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:46.680904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2514","reported_seq":2559,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.819127+0000","last_change":"2024-05-09T07:39:53.819156+0000","last_active":"2024-05-09T07:39:53.819127+0000","last_peered":"2024-05-09T07:39:53.819127+0000","last_clean":"2024-05-09T07:39:53.819127+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:39:53.819127+0000","last_undegraded":"2024-05-09T07:39:53.819127+0000","last_fullsized":"2024-05-09T07:39:53.819127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2514","last_scrub_stamp":"2024-05-09T07:39:53.819024+0000","last_deep_scrub":"27'2514","last_deep_scrub_stamp":"2024-05-09T07:39:53.819024+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.819024+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:11.925801+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2640","reported_seq":2685,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.899789+0000","last_change":"2024-05-09T07:39:49.899812+0000","last_active":"2024-05-09T07:39:49.899789+0000","last_peered":"2024-05-09T07:39:49.899789+0000","last_clean":"2024-05-09T07:39:49.899789+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:39:49.899789+0000","last_undegraded":"2024-05-09T07:39:49.899789+0000","last_fullsized":"2024-05-09T07:39:49.899789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2640","last_scrub_stamp":"2024-05-09T07:39:49.899700+0000","last_deep_scrub":"27'2640","last_deep_scrub_stamp":"2024-05-09T07:39:49.899700+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.899700+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:18.060341+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2612","reported_seq":2656,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.307290+0000","last_change":"2024-05-09T07:39:56.307324+0000","last_active":"2024-05-09T07:39:56.307290+0000","last_peered":"2024-05-09T07:39:56.307290+0000","last_clean":"2024-05-09T07:39:56.307290+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:39:56.307290+0000","last_undegraded":"2024-05-09T07:39:56.307290+0000","last_fullsized":"2024-05-09T07:39:56.307290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2612","last_scrub_stamp":"2024-05-09T07:39:56.307203+0000","last_deep_scrub":"27'2612","last_deep_scrub_stamp":"2024-05-09T07:39:56.307203+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.307203+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.072060+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2548","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.201784+0000","last_change":"2024-05-09T07:39:43.201825+0000","last_active":"2024-05-09T07:39:43.201784+0000","last_peered":"2024-05-09T07:39:43.201784+0000","last_clean":"2024-05-09T07:39:43.201784+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:39:43.201784+0000","last_undegraded":"2024-05-09T07:39:43.201784+0000","last_fullsized":"2024-05-09T07:39:43.201784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2548","last_scrub_stamp":"2024-05-09T07:39:43.201670+0000","last_deep_scrub":"27'2548","last_deep_scrub_stamp":"2024-05-09T07:39:43.201670+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.201670+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:19.053061+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2567","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.264382+0000","last_change":"2024-05-09T07:39:52.264408+0000","last_active":"2024-05-09T07:39:52.264382+0000","last_peered":"2024-05-09T07:39:52.264382+0000","last_clean":"2024-05-09T07:39:52.264382+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:39:52.264382+0000","last_undegraded":"2024-05-09T07:39:52.264382+0000","last_fullsized":"2024-05-09T07:39:52.264382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2567","last_scrub_stamp":"2024-05-09T07:39:52.264297+0000","last_deep_scrub":"27'2567","last_deep_scrub_stamp":"2024-05-09T07:39:52.264297+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.264297+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.371073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2520","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.965219+0000","last_change":"2024-05-09T07:39:52.965272+0000","last_active":"2024-05-09T07:39:52.965219+0000","last_peered":"2024-05-09T07:39:52.965219+0000","last_clean":"2024-05-09T07:39:52.965219+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:39:52.965219+0000","last_undegraded":"2024-05-09T07:39:52.965219+0000","last_fullsized":"2024-05-09T07:39:52.965219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2520","last_scrub_stamp":"2024-05-09T07:39:52.965125+0000","last_deep_scrub":"27'2520","last_deep_scrub_stamp":"2024-05-09T07:39:52.965125+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.965125+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:09.305340+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2550","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.264589+0000","last_change":"2024-05-09T07:39:55.264614+0000","last_active":"2024-05-09T07:39:55.264589+0000","last_peered":"2024-05-09T07:39:55.264589+0000","last_clean":"2024-05-09T07:39:55.264589+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:39:55.264589+0000","last_undegraded":"2024-05-09T07:39:55.264589+0000","last_fullsized":"2024-05-09T07:39:55.264589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2550","last_scrub_stamp":"2024-05-09T07:39:55.264504+0000","last_deep_scrub":"27'2550","last_deep_scrub_stamp":"2024-05-09T07:39:55.264504+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.264504+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:20.886229+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2592","reported_seq":2640,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.012614+0000","last_change":"2024-05-09T07:39:52.012614+0000","last_active":"2024-05-09T07:39:52.012614+0000","last_peered":"2024-05-09T07:39:52.012614+0000","last_clean":"2024-05-09T07:39:52.012614+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:39:52.012614+0000","last_undegraded":"2024-05-09T07:39:52.012614+0000","last_fullsized":"2024-05-09T07:39:52.012614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2592","last_scrub_stamp":"2024-05-09T07:39:52.012510+0000","last_deep_scrub":"27'2592","last_deep_scrub_stamp":"2024-05-09T07:39:52.012510+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.012510+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:37.844862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2671","reported_seq":2717,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.675862+0000","last_change":"2024-05-09T07:39:44.675901+0000","last_active":"2024-05-09T07:39:44.675862+0000","last_peered":"2024-05-09T07:39:44.675862+0000","last_clean":"2024-05-09T07:39:44.675862+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:39:44.675862+0000","last_undegraded":"2024-05-09T07:39:44.675862+0000","last_fullsized":"2024-05-09T07:39:44.675862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2671","last_scrub_stamp":"2024-05-09T07:39:44.675764+0000","last_deep_scrub":"27'2671","last_deep_scrub_stamp":"2024-05-09T07:39:44.675764+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.675764+0000","objects_scrubbed":0,"log_size":2671,"log_dups_size":0,"ondisk_log_size":2671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:44.946523+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2502","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.375269+0000","last_change":"2024-05-09T07:39:49.375292+0000","last_active":"2024-05-09T07:39:49.375269+0000","last_peered":"2024-05-09T07:39:49.375269+0000","last_clean":"2024-05-09T07:39:49.375269+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:39:49.375269+0000","last_undegraded":"2024-05-09T07:39:49.375269+0000","last_fullsized":"2024-05-09T07:39:49.375269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2502","last_scrub_stamp":"2024-05-09T07:39:49.375180+0000","last_deep_scrub":"27'2502","last_deep_scrub_stamp":"2024-05-09T07:39:49.375180+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.375180+0000","objects_scrubbed":0,"log_size":2502,"log_dups_size":0,"ondisk_log_size":2502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:40.237981+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2487","reported_seq":2533,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.290039+0000","last_change":"2024-05-09T07:39:57.290079+0000","last_active":"2024-05-09T07:39:57.290039+0000","last_peered":"2024-05-09T07:39:57.290039+0000","last_clean":"2024-05-09T07:39:57.290039+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:39:57.290039+0000","last_undegraded":"2024-05-09T07:39:57.290039+0000","last_fullsized":"2024-05-09T07:39:57.290039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2487","last_scrub_stamp":"2024-05-09T07:39:57.289935+0000","last_deep_scrub":"27'2487","last_deep_scrub_stamp":"2024-05-09T07:39:57.289935+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.289935+0000","objects_scrubbed":0,"log_size":2487,"log_dups_size":0,"ondisk_log_size":2487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:20.500903+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12270,"num_read_kb":3117350,"num_write":214738,"num_write_kb":30722215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232137,"ondisk_log_size":232137,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164829,"ondisk_log_size":164829,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":248714909,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":3182,"num_read_kb":19440,"num_write":211797,"num_write_kb":1695590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9301,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497491968,"data_stored":497429818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193015,"ondisk_log_size":193015,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575168,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":960860,"kb_used_data":55124,"kb_used_omap":1,"kb_used_meta":905726,"kb_avail":93410980,"statfs":{"total":96636764160,"available":95652843520,"internally_reserved":0,"allocated":56446976,"data_stored":56268948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":927463882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575167,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104412,"kb_used_data":66836,"kb_used_omap":1,"kb_used_meta":1037566,"kb_avail":93267428,"statfs":{"total":96636764160,"available":95505846272,"internally_reserved":0,"allocated":68440064,"data_stored":68261582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1062468038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575167,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":983016,"kb_used_data":45344,"kb_used_omap":1,"kb_used_meta":937662,"kb_avail":93388824,"statfs":{"total":96636764160,"available":95630155776,"internally_reserved":0,"allocated":46432256,"data_stored":46258861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960166341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575166,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1043696,"kb_used_data":65832,"kb_used_omap":1,"kb_used_meta":977854,"kb_avail":93328144,"statfs":{"total":96636764160,"available":95568019456,"internally_reserved":0,"allocated":67411968,"data_stored":67230417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001322950},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575168,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1220228,"kb_used_data":70140,"kb_used_omap":1,"kb_used_meta":1150078,"kb_avail":93151612,"statfs":{"total":96636764160,"available":95387250688,"internally_reserved":0,"allocated":71823360,"data_stored":71644151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1177680327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575169,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710908,"kb_used_data":53748,"kb_used_omap":1,"kb_used_meta":657150,"kb_avail":93660932,"statfs":{"total":96636764160,"available":95908794368,"internally_reserved":0,"allocated":55037952,"data_stored":54866263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":672922053},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575170,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1233900,"kb_used_data":65828,"kb_used_omap":1,"kb_used_meta":1168062,"kb_avail":93137940,"statfs":{"total":96636764160,"available":95373250560,"internally_reserved":0,"allocated":67407872,"data_stored":67230451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1196095941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575168,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":899828,"kb_used_data":66412,"kb_used_omap":1,"kb_used_meta":833406,"kb_avail":93472012,"statfs":{"total":96636764160,"available":95715340288,"internally_reserved":0,"allocated":68005888,"data_stored":67821297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":853408197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67109502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54747136,"data_stored":54744836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71532544,"data_stored":71522724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46141440,"data_stored":46137434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":68149248,"data_stored":68140155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":55562240,"data_stored":55557153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:40:04.849 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.849 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.850 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.850 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.850 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.850 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.851 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.851 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-05-09T07:13:48.478581+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=13, tm_sec=48, tm_wday=3, tm_yday=130, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=9, tm_hour=7, tm_min=39, tm_sec=39, tm_wday=3, tm_yday=130, tm_isdst=0) 2024-05-09T07:40:04.851 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-09T07:40:24.853 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T07:40:24.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.943+0000 7f0209f55640 1 Processor -- start 2024-05-09T07:40:24.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.943+0000 7f0209f55640 1 -- start start 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209f55640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02041524f0 0x7f0204152980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209f55640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f0204156c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209f55640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f02040c9ad0 con 0x7f02040c9f90 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209f55640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f0204152ec0 con 0x7f02041524f0 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209f55640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f02041553a0 con 0x7f0204156780 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0203fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02041524f0 0x7f0204152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0208f53640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f02040c9f90 0x7f02040cb100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52054/0 (socket says 172.21.15.160:52054) 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0208f53640 1 -- 172.21.15.160:0/1796534873 learned_addr learned my addr 172.21.15.160:0/1796534873 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0203fff640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f02041524f0 0x7f0204152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33892/0 (socket says 172.21.15.160:33892) 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209754640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f0204156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0203fff640 1 -- 172.21.15.160:0/1796534873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 msgr2=0x7f0204156c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:24.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0203fff640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f0204156c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0203fff640 1 -- 172.21.15.160:0/1796534873 >> v1:172.21.15.160:6789/0 conn(0x7f02040c9f90 legacy=0x7f02040cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0203fff640 1 -- 172.21.15.160:0/1796534873 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0204155680 con 0x7f02041524f0 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.944+0000 7f0209754640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f0204156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.945+0000 7f0203fff640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02041524f0 0x7f0204152980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01f00097a0 tx=0x7f01f002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541f12f0b7ae298d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.945+0000 7f02037fe640 1 -- 172.21.15.160:0/1796534873 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0037070 con 0x7f02041524f0 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.945+0000 7f02037fe640 1 -- 172.21.15.160:0/1796534873 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f002ae50 con 0x7f02041524f0 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.945+0000 7f02037fe640 1 -- 172.21.15.160:0/1796534873 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f0032660 con 0x7f02041524f0 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 -- 172.21.15.160:0/1796534873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02041524f0 msgr2=0x7f0204152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02041524f0 0x7f0204152980 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f01f00097a0 tx=0x7f01f002a100 comp rx=0 tx=0).stop 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 -- 172.21.15.160:0/1796534873 shutdown_connections 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f0204156c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 --2- 172.21.15.160:0/1796534873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02041524f0 0x7f0204152980 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 -- 172.21.15.160:0/1796534873 >> 172.21.15.160:0/1796534873 conn(0x7f0204072cb0 msgr2=0x7f02040730c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 -- 172.21.15.160:0/1796534873 shutdown_connections 2024-05-09T07:40:24.958 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.946+0000 7f0209f55640 1 -- 172.21.15.160:0/1796534873 wait complete. 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.947+0000 7f0209f55640 1 Processor -- start 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.947+0000 7f0209f55640 1 -- start start 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 0x7f02041eacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02041524f0 0x7f02041ee950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f020414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f02040cb810 con 0x7f02040c9f90 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f0204155c40 con 0x7f02041524f0 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f02040cb990 con 0x7f0204156780 2024-05-09T07:40:24.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 0x7f02041eacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 0x7f02041eacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33900/0 (socket says 172.21.15.160:33900) 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 -- 172.21.15.160:0/458913547 learned_addr learned my addr 172.21.15.160:0/458913547 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209754640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f020414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 -- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 msgr2=0x7f020414df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f020414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0203fff640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02041524f0 0x7f02041ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 -- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02041524f0 msgr2=0x7f02041ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02041524f0 0x7f02041ee950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 -- 172.21.15.160:0/458913547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02041ebc40 con 0x7f02040c9f90 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0208f53640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 0x7f02041eacb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c3c0 tx=0x7f01f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8017070 con 0x7f02040c9f90 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01f8005ce0 con 0x7f02040c9f90 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.948+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02040aa8c0 con 0x7f02040c9f90 2024-05-09T07:40:24.962 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.949+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01f8007310 con 0x7f02040c9f90 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.949+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041ee430 con 0x7f02040c9f90 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.950+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02041563c0 con 0x7f02040c9f90 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.950+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f01f801a070 con 0x7f02040c9f90 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.951+0000 7f02017fa640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f01e806be30 0x7f01e806e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.951+0000 7f0203fff640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f01e806be30 0x7f01e806e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.951+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f01f808a550 con 0x7f02040c9f90 2024-05-09T07:40:24.963 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.951+0000 7f0203fff640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f01e806be30 0x7f01e806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01f0034000 tx=0x7f01f002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:24.966 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:24.955+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f02041563c0 con 0x7f02040c9f90 2024-05-09T07:40:25.137 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.126+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 --> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02040aa670 con 0x7f01e806be30 2024-05-09T07:40:25.145 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.134+0000 7f02017fa640 1 -- 172.21.15.160:0/458913547 <== mgr.4099 v2:172.21.15.160:6832/2408879908 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+336733 (secure 0 0 0) 0x7f02040aa670 con 0x7f01e806be30 2024-05-09T07:40:25.149 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:40:25.152 INFO:teuthology.orchestra.run.smithi160.stderr:dumped all 2024-05-09T07:40:25.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f01e806be30 msgr2=0x7f01e806e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f01e806be30 0x7f01e806e2f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f01f0034000 tx=0x7f01f002e000 comp rx=0 tx=0).stop 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 msgr2=0x7f02041eacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 0x7f02041eacb0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f01f800c3c0 tx=0x7f01f800c880 comp rx=0 tx=0).stop 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 shutdown_connections 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f01e806be30 0x7f01e806e2f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f02041524f0 0x7f02041ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f0204156780 0x7f020414df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 --2- 172.21.15.160:0/458913547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f02040c9f90 0x7f02041eacb0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 >> 172.21.15.160:0/458913547 conn(0x7f0204072cb0 msgr2=0x7f02041531e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 shutdown_connections 2024-05-09T07:40:25.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.138+0000 7f0209f55640 1 -- 172.21.15.160:0/458913547 wait complete. 2024-05-09T07:40:25.160 INFO:teuthology.orchestra.run.smithi160.stdout:{"pg_ready":true,"pg_map":{"version":825,"stamp":"2024-05-09T07:40:23.571436+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":249305277,"num_objects":85,"num_object_clones":0,"num_object_copies":170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85,"num_whiteouts":0,"num_read":15568,"num_read_kb":3136890,"num_write":427814,"num_write_kb":32420221,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9301,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":590100,"ondisk_log_size":590100,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":8156848,"kb_used_data":489264,"kb_used_omap":12,"kb_used_meta":7667507,"kb_avail":746817872,"statfs":{"total":773094113280,"available":764741500928,"internally_reserved":0,"allocated":501006336,"data_stored":499581970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":7851527729},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002592"},"pg_stats":[{"pgid":"2.1f","version":"27'2677","reported_seq":2725,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.638587+0000","last_change":"2024-05-09T07:39:43.638613+0000","last_active":"2024-05-09T07:39:43.638587+0000","last_peered":"2024-05-09T07:39:43.638587+0000","last_clean":"2024-05-09T07:39:43.638587+0000","last_became_active":"2024-05-09T07:13:44.474519+0000","last_became_peered":"2024-05-09T07:13:44.474519+0000","last_unstale":"2024-05-09T07:39:43.638587+0000","last_undegraded":"2024-05-09T07:39:43.638587+0000","last_fullsized":"2024-05-09T07:39:43.638587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2677","last_scrub_stamp":"2024-05-09T07:39:43.638526+0000","last_deep_scrub":"27'2677","last_deep_scrub_stamp":"2024-05-09T07:39:43.638526+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.638526+0000","objects_scrubbed":2,"log_size":2677,"log_dups_size":0,"ondisk_log_size":2677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:03.343390+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":2687,"num_write_kb":18241,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":561,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2578","reported_seq":2626,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.229724+0000","last_change":"2024-05-09T07:39:46.229724+0000","last_active":"2024-05-09T07:39:46.229724+0000","last_peered":"2024-05-09T07:39:46.229724+0000","last_clean":"2024-05-09T07:39:46.229724+0000","last_became_active":"2024-05-09T07:13:45.491922+0000","last_became_peered":"2024-05-09T07:13:45.491922+0000","last_unstale":"2024-05-09T07:39:46.229724+0000","last_undegraded":"2024-05-09T07:39:46.229724+0000","last_fullsized":"2024-05-09T07:39:46.229724+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2578","last_scrub_stamp":"2024-05-09T07:39:46.229628+0000","last_deep_scrub":"27'2578","last_deep_scrub_stamp":"2024-05-09T07:39:46.229628+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.229628+0000","objects_scrubbed":0,"log_size":2578,"log_dups_size":0,"ondisk_log_size":2578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:51.406925+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'9823","reported_seq":10262,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:02.356096+0000","last_change":"2024-05-09T07:40:02.356121+0000","last_active":"2024-05-09T07:40:02.356096+0000","last_peered":"2024-05-09T07:40:02.356096+0000","last_clean":"2024-05-09T07:40:02.356096+0000","last_became_active":"2024-05-09T07:13:49.940619+0000","last_became_peered":"2024-05-09T07:13:49.940619+0000","last_unstale":"2024-05-09T07:40:02.356096+0000","last_undegraded":"2024-05-09T07:40:02.356096+0000","last_fullsized":"2024-05-09T07:40:02.356096+0000","mapping_epoch":22,"log_start":"27'3504","ondisk_log_start":"27'3504","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9823","last_scrub_stamp":"2024-05-09T07:40:02.356010+0000","last_deep_scrub":"27'9823","last_deep_scrub_stamp":"2024-05-09T07:40:02.356010+0000","last_clean_scrub_stamp":"2024-05-09T07:40:02.356010+0000","objects_scrubbed":0,"log_size":6319,"log_dups_size":0,"ondisk_log_size":6319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:24:08.745199+0000","scrub_duration":46,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":399,"num_read_kb":101346,"num_write":7176,"num_write_kb":1009567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'7790","reported_seq":7840,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.239196+0000","last_change":"2024-05-09T07:39:48.239196+0000","last_active":"2024-05-09T07:39:48.239196+0000","last_peered":"2024-05-09T07:39:48.239196+0000","last_clean":"2024-05-09T07:39:48.239196+0000","last_became_active":"2024-05-09T07:13:44.460416+0000","last_became_peered":"2024-05-09T07:13:44.460416+0000","last_unstale":"2024-05-09T07:39:48.239196+0000","last_undegraded":"2024-05-09T07:39:48.239196+0000","last_fullsized":"2024-05-09T07:39:48.239196+0000","mapping_epoch":17,"log_start":"27'1500","ondisk_log_start":"27'1500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7790","last_scrub_stamp":"2024-05-09T07:39:48.239088+0000","last_deep_scrub":"27'7790","last_deep_scrub_stamp":"2024-05-09T07:39:48.239088+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.239088+0000","objects_scrubbed":1,"log_size":6290,"log_dups_size":0,"ondisk_log_size":6290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:06.345864+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":7802,"num_write_kb":54453,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'2558","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.586739+0000","last_change":"2024-05-09T07:39:50.586762+0000","last_active":"2024-05-09T07:39:50.586739+0000","last_peered":"2024-05-09T07:39:50.586739+0000","last_clean":"2024-05-09T07:39:50.586739+0000","last_became_active":"2024-05-09T07:13:45.488992+0000","last_became_peered":"2024-05-09T07:13:45.488992+0000","last_unstale":"2024-05-09T07:39:50.586739+0000","last_undegraded":"2024-05-09T07:39:50.586739+0000","last_fullsized":"2024-05-09T07:39:50.586739+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-05-09T07:39:50.586658+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-05-09T07:39:50.586658+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.586658+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:36.419009+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'9533","reported_seq":9929,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.351870+0000","last_change":"2024-05-09T07:39:50.351903+0000","last_active":"2024-05-09T07:39:50.351870+0000","last_peered":"2024-05-09T07:39:50.351870+0000","last_clean":"2024-05-09T07:39:50.351870+0000","last_became_active":"2024-05-09T07:13:49.505406+0000","last_became_peered":"2024-05-09T07:13:49.505406+0000","last_unstale":"2024-05-09T07:39:50.351870+0000","last_undegraded":"2024-05-09T07:39:50.351870+0000","last_fullsized":"2024-05-09T07:39:50.351870+0000","mapping_epoch":22,"log_start":"27'2329","ondisk_log_start":"27'2329","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9533","last_scrub_stamp":"2024-05-09T07:39:50.351773+0000","last_deep_scrub":"27'9533","last_deep_scrub_stamp":"2024-05-09T07:39:50.351773+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.351773+0000","objects_scrubbed":0,"log_size":7204,"log_dups_size":0,"ondisk_log_size":7204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:41.214574+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6761,"num_write_kb":1095702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'9533","reported_seq":9720,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.436111+0000","last_change":"2024-05-09T07:39:46.436150+0000","last_active":"2024-05-09T07:39:46.436111+0000","last_peered":"2024-05-09T07:39:46.436111+0000","last_clean":"2024-05-09T07:39:46.436111+0000","last_became_active":"2024-05-09T07:13:44.474059+0000","last_became_peered":"2024-05-09T07:13:44.474059+0000","last_unstale":"2024-05-09T07:39:46.436111+0000","last_undegraded":"2024-05-09T07:39:46.436111+0000","last_fullsized":"2024-05-09T07:39:46.436111+0000","mapping_epoch":17,"log_start":"27'2500","ondisk_log_start":"27'2500","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9533","last_scrub_stamp":"2024-05-09T07:39:46.436028+0000","last_deep_scrub":"27'9533","last_deep_scrub_stamp":"2024-05-09T07:39:46.436028+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.436028+0000","objects_scrubbed":3,"log_size":7033,"log_dups_size":0,"ondisk_log_size":7033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:53.364308+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5224923,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":143,"num_read_kb":1099,"num_write":9529,"num_write_kb":77058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'2666","reported_seq":2711,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.987062+0000","last_change":"2024-05-09T07:39:55.987094+0000","last_active":"2024-05-09T07:39:55.987062+0000","last_peered":"2024-05-09T07:39:55.987062+0000","last_clean":"2024-05-09T07:39:55.987062+0000","last_became_active":"2024-05-09T07:13:45.491292+0000","last_became_peered":"2024-05-09T07:13:45.491292+0000","last_unstale":"2024-05-09T07:39:55.987062+0000","last_undegraded":"2024-05-09T07:39:55.987062+0000","last_fullsized":"2024-05-09T07:39:55.987062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2666","last_scrub_stamp":"2024-05-09T07:39:55.986977+0000","last_deep_scrub":"27'2666","last_deep_scrub_stamp":"2024-05-09T07:39:55.986977+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.986977+0000","objects_scrubbed":0,"log_size":2666,"log_dups_size":0,"ondisk_log_size":2666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:19.197944+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'9072","reported_seq":9490,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.301275+0000","last_change":"2024-05-09T07:39:53.301303+0000","last_active":"2024-05-09T07:39:53.301275+0000","last_peered":"2024-05-09T07:39:53.301275+0000","last_clean":"2024-05-09T07:39:53.301275+0000","last_became_active":"2024-05-09T07:13:49.507071+0000","last_became_peered":"2024-05-09T07:13:49.507071+0000","last_unstale":"2024-05-09T07:39:53.301275+0000","last_undegraded":"2024-05-09T07:39:53.301275+0000","last_fullsized":"2024-05-09T07:39:53.301275+0000","mapping_epoch":22,"log_start":"27'1901","ondisk_log_start":"27'1901","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9072","last_scrub_stamp":"2024-05-09T07:39:53.301167+0000","last_deep_scrub":"27'9072","last_deep_scrub_stamp":"2024-05-09T07:39:53.301167+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.301167+0000","objects_scrubbed":0,"log_size":7171,"log_dups_size":0,"ondisk_log_size":7171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:09.641382+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":378,"num_read_kb":96012,"num_write":6307,"num_write_kb":900735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'4294","reported_seq":4340,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.294687+0000","last_change":"2024-05-09T07:39:51.294742+0000","last_active":"2024-05-09T07:39:51.294687+0000","last_peered":"2024-05-09T07:39:51.294687+0000","last_clean":"2024-05-09T07:39:51.294687+0000","last_became_active":"2024-05-09T07:13:44.475886+0000","last_became_peered":"2024-05-09T07:13:44.475886+0000","last_unstale":"2024-05-09T07:39:51.294687+0000","last_undegraded":"2024-05-09T07:39:51.294687+0000","last_fullsized":"2024-05-09T07:39:51.294687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4294","last_scrub_stamp":"2024-05-09T07:39:51.294570+0000","last_deep_scrub":"27'4294","last_deep_scrub_stamp":"2024-05-09T07:39:51.294570+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.294570+0000","objects_scrubbed":2,"log_size":4294,"log_dups_size":0,"ondisk_log_size":4294,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:21.037947+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":0,"num_write":4306,"num_write_kb":47774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'2602","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.062040+0000","last_change":"2024-05-09T07:39:45.062063+0000","last_active":"2024-05-09T07:39:45.062040+0000","last_peered":"2024-05-09T07:39:45.062040+0000","last_clean":"2024-05-09T07:39:45.062040+0000","last_became_active":"2024-05-09T07:13:45.487262+0000","last_became_peered":"2024-05-09T07:13:45.487262+0000","last_unstale":"2024-05-09T07:39:45.062040+0000","last_undegraded":"2024-05-09T07:39:45.062040+0000","last_fullsized":"2024-05-09T07:39:45.062040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2602","last_scrub_stamp":"2024-05-09T07:39:45.061935+0000","last_deep_scrub":"27'2602","last_deep_scrub_stamp":"2024-05-09T07:39:45.061935+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.061935+0000","objects_scrubbed":0,"log_size":2602,"log_dups_size":0,"ondisk_log_size":2602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:51.990215+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'9250","reported_seq":9732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.357996+0000","last_change":"2024-05-09T07:39:52.358022+0000","last_active":"2024-05-09T07:39:52.357996+0000","last_peered":"2024-05-09T07:39:52.357996+0000","last_clean":"2024-05-09T07:39:52.357996+0000","last_became_active":"2024-05-09T07:13:49.506414+0000","last_became_peered":"2024-05-09T07:13:49.506414+0000","last_unstale":"2024-05-09T07:39:52.357996+0000","last_undegraded":"2024-05-09T07:39:52.357996+0000","last_fullsized":"2024-05-09T07:39:52.357996+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9250","last_scrub_stamp":"2024-05-09T07:39:52.357879+0000","last_deep_scrub":"27'9250","last_deep_scrub_stamp":"2024-05-09T07:39:52.357879+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.357879+0000","objects_scrubbed":0,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:38.190231+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":442,"num_read_kb":112268,"num_write":6539,"num_write_kb":853848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'2585","reported_seq":2632,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.856177+0000","last_change":"2024-05-09T07:39:50.856177+0000","last_active":"2024-05-09T07:39:50.856177+0000","last_peered":"2024-05-09T07:39:50.856177+0000","last_clean":"2024-05-09T07:39:50.856177+0000","last_became_active":"2024-05-09T07:13:45.487018+0000","last_became_peered":"2024-05-09T07:13:45.487018+0000","last_unstale":"2024-05-09T07:39:50.856177+0000","last_undegraded":"2024-05-09T07:39:50.856177+0000","last_fullsized":"2024-05-09T07:39:50.856177+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2585","last_scrub_stamp":"2024-05-09T07:39:50.856027+0000","last_deep_scrub":"27'2585","last_deep_scrub_stamp":"2024-05-09T07:39:50.856027+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.856027+0000","objects_scrubbed":0,"log_size":2585,"log_dups_size":0,"ondisk_log_size":2585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:49.201658+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'6970","reported_seq":7017,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.419458+0000","last_change":"2024-05-09T07:39:47.419494+0000","last_active":"2024-05-09T07:39:47.419458+0000","last_peered":"2024-05-09T07:39:47.419458+0000","last_clean":"2024-05-09T07:39:47.419458+0000","last_became_active":"2024-05-09T07:13:44.474047+0000","last_became_peered":"2024-05-09T07:13:44.474047+0000","last_unstale":"2024-05-09T07:39:47.419458+0000","last_undegraded":"2024-05-09T07:39:47.419458+0000","last_fullsized":"2024-05-09T07:39:47.419458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6970","last_scrub_stamp":"2024-05-09T07:39:47.419366+0000","last_deep_scrub":"27'6970","last_deep_scrub_stamp":"2024-05-09T07:39:47.419366+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.419366+0000","objects_scrubbed":4,"log_size":6970,"log_dups_size":0,"ondisk_log_size":6970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:24.872236+0000","scrub_duration":45,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6974,"num_write_kb":53856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":334,"num_omap_keys":1,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'9808","reported_seq":10065,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.331939+0000","last_change":"2024-05-09T07:39:49.331965+0000","last_active":"2024-05-09T07:39:49.331939+0000","last_peered":"2024-05-09T07:39:49.331939+0000","last_clean":"2024-05-09T07:39:49.331939+0000","last_became_active":"2024-05-09T07:13:49.505004+0000","last_became_peered":"2024-05-09T07:13:49.505004+0000","last_unstale":"2024-05-09T07:39:49.331939+0000","last_undegraded":"2024-05-09T07:39:49.331939+0000","last_fullsized":"2024-05-09T07:39:49.331939+0000","mapping_epoch":22,"log_start":"27'2633","ondisk_log_start":"27'2633","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9808","last_scrub_stamp":"2024-05-09T07:39:49.331826+0000","last_deep_scrub":"27'9808","last_deep_scrub_stamp":"2024-05-09T07:39:49.331826+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.331826+0000","objects_scrubbed":0,"log_size":7175,"log_dups_size":0,"ondisk_log_size":7175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:54.509122+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":215,"num_read_kb":54610,"num_write":7041,"num_write_kb":1008697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'2616","reported_seq":2662,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.249456+0000","last_change":"2024-05-09T07:39:51.249479+0000","last_active":"2024-05-09T07:39:51.249456+0000","last_peered":"2024-05-09T07:39:51.249456+0000","last_clean":"2024-05-09T07:39:51.249456+0000","last_became_active":"2024-05-09T07:13:45.478669+0000","last_became_peered":"2024-05-09T07:13:45.478669+0000","last_unstale":"2024-05-09T07:39:51.249456+0000","last_undegraded":"2024-05-09T07:39:51.249456+0000","last_fullsized":"2024-05-09T07:39:51.249456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2616","last_scrub_stamp":"2024-05-09T07:39:51.249375+0000","last_deep_scrub":"27'2616","last_deep_scrub_stamp":"2024-05-09T07:39:51.249375+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.249375+0000","objects_scrubbed":0,"log_size":2616,"log_dups_size":0,"ondisk_log_size":2616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:16.871100+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'7784","reported_seq":7833,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.348520+0000","last_change":"2024-05-09T07:39:43.348520+0000","last_active":"2024-05-09T07:39:43.348520+0000","last_peered":"2024-05-09T07:39:43.348520+0000","last_clean":"2024-05-09T07:39:43.348520+0000","last_became_active":"2024-05-09T07:13:44.469310+0000","last_became_peered":"2024-05-09T07:13:44.469310+0000","last_unstale":"2024-05-09T07:39:43.348520+0000","last_undegraded":"2024-05-09T07:39:43.348520+0000","last_fullsized":"2024-05-09T07:39:43.348520+0000","mapping_epoch":17,"log_start":"27'400","ondisk_log_start":"27'400","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7784","last_scrub_stamp":"2024-05-09T07:39:43.348350+0000","last_deep_scrub":"27'7784","last_deep_scrub_stamp":"2024-05-09T07:39:43.348350+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.348350+0000","objects_scrubbed":2,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:49.480366+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":58435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'9632","reported_seq":10181,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.234971+0000","last_change":"2024-05-09T07:39:55.234999+0000","last_active":"2024-05-09T07:39:55.234971+0000","last_peered":"2024-05-09T07:39:55.234971+0000","last_clean":"2024-05-09T07:39:55.234971+0000","last_became_active":"2024-05-09T07:13:49.507309+0000","last_became_peered":"2024-05-09T07:13:49.507309+0000","last_unstale":"2024-05-09T07:39:55.234971+0000","last_undegraded":"2024-05-09T07:39:55.234971+0000","last_fullsized":"2024-05-09T07:39:55.234971+0000","mapping_epoch":22,"log_start":"27'2443","ondisk_log_start":"27'2443","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9632","last_scrub_stamp":"2024-05-09T07:39:55.234858+0000","last_deep_scrub":"27'9632","last_deep_scrub_stamp":"2024-05-09T07:39:55.234858+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.234858+0000","objects_scrubbed":0,"log_size":7189,"log_dups_size":0,"ondisk_log_size":7189,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:31.851464+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":508,"num_read_kb":129802,"num_write":6852,"num_write_kb":970185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'2685","reported_seq":2732,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.276740+0000","last_change":"2024-05-09T07:39:53.276740+0000","last_active":"2024-05-09T07:39:53.276740+0000","last_peered":"2024-05-09T07:39:53.276740+0000","last_clean":"2024-05-09T07:39:53.276740+0000","last_became_active":"2024-05-09T07:13:45.479496+0000","last_became_peered":"2024-05-09T07:13:45.479496+0000","last_unstale":"2024-05-09T07:39:53.276740+0000","last_undegraded":"2024-05-09T07:39:53.276740+0000","last_fullsized":"2024-05-09T07:39:53.276740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2685","last_scrub_stamp":"2024-05-09T07:39:53.276635+0000","last_deep_scrub":"27'2685","last_deep_scrub_stamp":"2024-05-09T07:39:53.276635+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.276635+0000","objects_scrubbed":0,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:16.487602+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'3775","reported_seq":3824,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.347239+0000","last_change":"2024-05-09T07:39:47.347239+0000","last_active":"2024-05-09T07:39:47.347239+0000","last_peered":"2024-05-09T07:39:47.347239+0000","last_clean":"2024-05-09T07:39:47.347239+0000","last_became_active":"2024-05-09T07:13:44.475524+0000","last_became_peered":"2024-05-09T07:13:44.475524+0000","last_unstale":"2024-05-09T07:39:47.347239+0000","last_undegraded":"2024-05-09T07:39:47.347239+0000","last_fullsized":"2024-05-09T07:39:47.347239+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3775","last_scrub_stamp":"2024-05-09T07:39:47.347152+0000","last_deep_scrub":"27'3775","last_deep_scrub_stamp":"2024-05-09T07:39:47.347152+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.347152+0000","objects_scrubbed":0,"log_size":3775,"log_dups_size":0,"ondisk_log_size":3775,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:45.692783+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3772,"num_write_kb":27117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'9053","reported_seq":9674,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.780347+0000","last_change":"2024-05-09T07:39:52.780393+0000","last_active":"2024-05-09T07:39:52.780347+0000","last_peered":"2024-05-09T07:39:52.780347+0000","last_clean":"2024-05-09T07:39:52.780347+0000","last_became_active":"2024-05-09T07:14:26.646222+0000","last_became_peered":"2024-05-09T07:14:26.646222+0000","last_unstale":"2024-05-09T07:39:52.780347+0000","last_undegraded":"2024-05-09T07:39:52.780347+0000","last_fullsized":"2024-05-09T07:39:52.780347+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9053","last_scrub_stamp":"2024-05-09T07:39:52.780216+0000","last_deep_scrub":"27'9053","last_deep_scrub_stamp":"2024-05-09T07:39:52.780216+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.780216+0000","objects_scrubbed":0,"log_size":9053,"log_dups_size":0,"ondisk_log_size":9053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:19:59.708496+0000","scrub_duration":44,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":589,"num_read_kb":149606,"num_write":6377,"num_write_kb":893211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4,7],"acting":[2,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'2588","reported_seq":2634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.267264+0000","last_change":"2024-05-09T07:39:46.267264+0000","last_active":"2024-05-09T07:39:46.267264+0000","last_peered":"2024-05-09T07:39:46.267264+0000","last_clean":"2024-05-09T07:39:46.267264+0000","last_became_active":"2024-05-09T07:13:45.481454+0000","last_became_peered":"2024-05-09T07:13:45.481454+0000","last_unstale":"2024-05-09T07:39:46.267264+0000","last_undegraded":"2024-05-09T07:39:46.267264+0000","last_fullsized":"2024-05-09T07:39:46.267264+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2588","last_scrub_stamp":"2024-05-09T07:39:46.267114+0000","last_deep_scrub":"27'2588","last_deep_scrub_stamp":"2024-05-09T07:39:46.267114+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.267114+0000","objects_scrubbed":0,"log_size":2588,"log_dups_size":0,"ondisk_log_size":2588,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:22.118506+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'2268","reported_seq":2316,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.298732+0000","last_change":"2024-05-09T07:39:49.298732+0000","last_active":"2024-05-09T07:39:49.298732+0000","last_peered":"2024-05-09T07:39:49.298732+0000","last_clean":"2024-05-09T07:39:49.298732+0000","last_became_active":"2024-05-09T07:13:44.474952+0000","last_became_peered":"2024-05-09T07:13:44.474952+0000","last_unstale":"2024-05-09T07:39:49.298732+0000","last_undegraded":"2024-05-09T07:39:49.298732+0000","last_fullsized":"2024-05-09T07:39:49.298732+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2268","last_scrub_stamp":"2024-05-09T07:39:49.298603+0000","last_deep_scrub":"27'2268","last_deep_scrub_stamp":"2024-05-09T07:39:49.298603+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.298603+0000","objects_scrubbed":1,"log_size":2268,"log_dups_size":0,"ondisk_log_size":2268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:30.124305+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2276,"num_write_kb":21988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'9931","reported_seq":10076,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.276205+0000","last_change":"2024-05-09T07:39:54.276230+0000","last_active":"2024-05-09T07:39:54.276205+0000","last_peered":"2024-05-09T07:39:54.276205+0000","last_clean":"2024-05-09T07:39:54.276205+0000","last_became_active":"2024-05-09T07:13:49.508027+0000","last_became_peered":"2024-05-09T07:13:49.508027+0000","last_unstale":"2024-05-09T07:39:54.276205+0000","last_undegraded":"2024-05-09T07:39:54.276205+0000","last_fullsized":"2024-05-09T07:39:54.276205+0000","mapping_epoch":22,"log_start":"27'2728","ondisk_log_start":"27'2728","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9931","last_scrub_stamp":"2024-05-09T07:39:54.276122+0000","last_deep_scrub":"27'9931","last_deep_scrub_stamp":"2024-05-09T07:39:54.276122+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.276122+0000","objects_scrubbed":0,"log_size":7203,"log_dups_size":0,"ondisk_log_size":7203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:51.582863+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":104,"num_read_kb":26416,"num_write":7236,"num_write_kb":974347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'2542","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.617914+0000","last_change":"2024-05-09T07:39:49.617914+0000","last_active":"2024-05-09T07:39:49.617914+0000","last_peered":"2024-05-09T07:39:49.617914+0000","last_clean":"2024-05-09T07:39:49.617914+0000","last_became_active":"2024-05-09T07:13:45.477114+0000","last_became_peered":"2024-05-09T07:13:45.477114+0000","last_unstale":"2024-05-09T07:39:49.617914+0000","last_undegraded":"2024-05-09T07:39:49.617914+0000","last_fullsized":"2024-05-09T07:39:49.617914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2542","last_scrub_stamp":"2024-05-09T07:39:49.617750+0000","last_deep_scrub":"27'2542","last_deep_scrub_stamp":"2024-05-09T07:39:49.617750+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.617750+0000","objects_scrubbed":0,"log_size":2542,"log_dups_size":0,"ondisk_log_size":2542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:07.724527+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'4070","reported_seq":4119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.940913+0000","last_change":"2024-05-09T07:39:47.940984+0000","last_active":"2024-05-09T07:39:47.940913+0000","last_peered":"2024-05-09T07:39:47.940913+0000","last_clean":"2024-05-09T07:39:47.940913+0000","last_became_active":"2024-05-09T07:13:44.474320+0000","last_became_peered":"2024-05-09T07:13:44.474320+0000","last_unstale":"2024-05-09T07:39:47.940913+0000","last_undegraded":"2024-05-09T07:39:47.940913+0000","last_fullsized":"2024-05-09T07:39:47.940913+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4070","last_scrub_stamp":"2024-05-09T07:39:47.940825+0000","last_deep_scrub":"27'4070","last_deep_scrub_stamp":"2024-05-09T07:39:47.940825+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.940825+0000","objects_scrubbed":4,"log_size":4070,"log_dups_size":0,"ondisk_log_size":4070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:07.645688+0000","scrub_duration":38,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4058,"num_write_kb":39474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'9249","reported_seq":9676,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.265154+0000","last_change":"2024-05-09T07:39:56.265183+0000","last_active":"2024-05-09T07:39:56.265154+0000","last_peered":"2024-05-09T07:39:56.265154+0000","last_clean":"2024-05-09T07:39:56.265154+0000","last_became_active":"2024-05-09T07:13:49.941029+0000","last_became_peered":"2024-05-09T07:13:49.941029+0000","last_unstale":"2024-05-09T07:39:56.265154+0000","last_undegraded":"2024-05-09T07:39:56.265154+0000","last_fullsized":"2024-05-09T07:39:56.265154+0000","mapping_epoch":22,"log_start":"27'2022","ondisk_log_start":"27'2022","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9249","last_scrub_stamp":"2024-05-09T07:39:56.265056+0000","last_deep_scrub":"27'9249","last_deep_scrub_stamp":"2024-05-09T07:39:56.265056+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.265056+0000","objects_scrubbed":0,"log_size":7227,"log_dups_size":0,"ondisk_log_size":7227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:19.476023+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":385,"num_read_kb":97790,"num_write":6537,"num_write_kb":843792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'2541","reported_seq":2588,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.300176+0000","last_change":"2024-05-09T07:39:56.300199+0000","last_active":"2024-05-09T07:39:56.300176+0000","last_peered":"2024-05-09T07:39:56.300176+0000","last_clean":"2024-05-09T07:39:56.300176+0000","last_became_active":"2024-05-09T07:13:45.493046+0000","last_became_peered":"2024-05-09T07:13:45.493046+0000","last_unstale":"2024-05-09T07:39:56.300176+0000","last_undegraded":"2024-05-09T07:39:56.300176+0000","last_fullsized":"2024-05-09T07:39:56.300176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2541","last_scrub_stamp":"2024-05-09T07:39:56.300098+0000","last_deep_scrub":"27'2541","last_deep_scrub_stamp":"2024-05-09T07:39:56.300098+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.300098+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:27.151559+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'6998","reported_seq":7654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.402653+0000","last_change":"2024-05-09T07:39:42.402653+0000","last_active":"2024-05-09T07:39:42.402653+0000","last_peered":"2024-05-09T07:39:42.402653+0000","last_clean":"2024-05-09T07:39:42.402653+0000","last_became_active":"2024-05-09T07:13:44.469335+0000","last_became_peered":"2024-05-09T07:13:44.469335+0000","last_unstale":"2024-05-09T07:39:42.402653+0000","last_undegraded":"2024-05-09T07:39:42.402653+0000","last_fullsized":"2024-05-09T07:39:42.402653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6998","last_scrub_stamp":"2024-05-09T07:39:42.402522+0000","last_deep_scrub":"27'6998","last_deep_scrub_stamp":"2024-05-09T07:39:42.402522+0000","last_clean_scrub_stamp":"2024-05-09T07:39:42.402522+0000","objects_scrubbed":3,"log_size":6998,"log_dups_size":0,"ondisk_log_size":6998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:53.052513+0000","scrub_duration":85,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":608,"num_read_kb":4487,"num_write":9817,"num_write_kb":61142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'9154","reported_seq":9347,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:59.331400+0000","last_change":"2024-05-09T07:39:59.331400+0000","last_active":"2024-05-09T07:39:59.331400+0000","last_peered":"2024-05-09T07:39:59.331400+0000","last_clean":"2024-05-09T07:39:59.331400+0000","last_became_active":"2024-05-09T07:13:49.938946+0000","last_became_peered":"2024-05-09T07:13:49.938946+0000","last_unstale":"2024-05-09T07:39:59.331400+0000","last_undegraded":"2024-05-09T07:39:59.331400+0000","last_fullsized":"2024-05-09T07:39:59.331400+0000","mapping_epoch":22,"log_start":"27'2834","ondisk_log_start":"27'2834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9154","last_scrub_stamp":"2024-05-09T07:39:59.331287+0000","last_deep_scrub":"27'9154","last_deep_scrub_stamp":"2024-05-09T07:39:59.331287+0000","last_clean_scrub_stamp":"2024-05-09T07:39:59.331287+0000","objects_scrubbed":0,"log_size":6320,"log_dups_size":0,"ondisk_log_size":6320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:11:51.681724+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":150,"num_read_kb":38100,"num_write":6430,"num_write_kb":884596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'2640","reported_seq":2687,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.354485+0000","last_change":"2024-05-09T07:39:52.354509+0000","last_active":"2024-05-09T07:39:52.354485+0000","last_peered":"2024-05-09T07:39:52.354485+0000","last_clean":"2024-05-09T07:39:52.354485+0000","last_became_active":"2024-05-09T07:13:45.478730+0000","last_became_peered":"2024-05-09T07:13:45.478730+0000","last_unstale":"2024-05-09T07:39:52.354485+0000","last_undegraded":"2024-05-09T07:39:52.354485+0000","last_fullsized":"2024-05-09T07:39:52.354485+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2640","last_scrub_stamp":"2024-05-09T07:39:52.354398+0000","last_deep_scrub":"27'2640","last_deep_scrub_stamp":"2024-05-09T07:39:52.354398+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.354398+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:06:18.855399+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'6526","reported_seq":6574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.358383+0000","last_change":"2024-05-09T07:39:48.358414+0000","last_active":"2024-05-09T07:39:48.358383+0000","last_peered":"2024-05-09T07:39:48.358383+0000","last_clean":"2024-05-09T07:39:48.358383+0000","last_became_active":"2024-05-09T07:13:44.474780+0000","last_became_peered":"2024-05-09T07:13:44.474780+0000","last_unstale":"2024-05-09T07:39:48.358383+0000","last_undegraded":"2024-05-09T07:39:48.358383+0000","last_fullsized":"2024-05-09T07:39:48.358383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6526","last_scrub_stamp":"2024-05-09T07:39:48.358312+0000","last_deep_scrub":"27'6526","last_deep_scrub_stamp":"2024-05-09T07:39:48.358312+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.358312+0000","objects_scrubbed":3,"log_size":6526,"log_dups_size":0,"ondisk_log_size":6526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:24.209704+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":6527,"num_write_kb":53434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'9270","reported_seq":10165,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.576205+0000","last_change":"2024-05-09T07:39:56.576230+0000","last_active":"2024-05-09T07:39:56.576205+0000","last_peered":"2024-05-09T07:39:56.576205+0000","last_clean":"2024-05-09T07:39:56.576205+0000","last_became_active":"2024-05-09T07:13:49.938804+0000","last_became_peered":"2024-05-09T07:13:49.938804+0000","last_unstale":"2024-05-09T07:39:56.576205+0000","last_undegraded":"2024-05-09T07:39:56.576205+0000","last_fullsized":"2024-05-09T07:39:56.576205+0000","mapping_epoch":22,"log_start":"27'1856","ondisk_log_start":"27'1856","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9270","last_scrub_stamp":"2024-05-09T07:39:56.576122+0000","last_deep_scrub":"27'9270","last_deep_scrub_stamp":"2024-05-09T07:39:56.576122+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.576122+0000","objects_scrubbed":0,"log_size":7414,"log_dups_size":0,"ondisk_log_size":7414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:59.746690+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":853,"num_read_kb":216662,"num_write":6474,"num_write_kb":849337,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'2527","reported_seq":2558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.748842+0000","last_change":"2024-05-09T07:39:50.748870+0000","last_active":"2024-05-09T07:39:50.748842+0000","last_peered":"2024-05-09T07:39:50.748842+0000","last_clean":"2024-05-09T07:39:50.748842+0000","last_became_active":"2024-05-09T07:14:26.645530+0000","last_became_peered":"2024-05-09T07:14:26.645530+0000","last_unstale":"2024-05-09T07:39:50.748842+0000","last_undegraded":"2024-05-09T07:39:50.748842+0000","last_fullsized":"2024-05-09T07:39:50.748842+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2527","last_scrub_stamp":"2024-05-09T07:39:50.748763+0000","last_deep_scrub":"27'2527","last_deep_scrub_stamp":"2024-05-09T07:39:50.748763+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.748763+0000","objects_scrubbed":0,"log_size":2527,"log_dups_size":0,"ondisk_log_size":2527,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:49:01.398753+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"27'8380","reported_seq":8429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.430910+0000","last_change":"2024-05-09T07:39:44.430951+0000","last_active":"2024-05-09T07:39:44.430910+0000","last_peered":"2024-05-09T07:39:44.430910+0000","last_clean":"2024-05-09T07:39:44.430910+0000","last_became_active":"2024-05-09T07:13:44.469431+0000","last_became_peered":"2024-05-09T07:13:44.469431+0000","last_unstale":"2024-05-09T07:39:44.430910+0000","last_undegraded":"2024-05-09T07:39:44.430910+0000","last_fullsized":"2024-05-09T07:39:44.430910+0000","mapping_epoch":17,"log_start":"27'1000","ondisk_log_start":"27'1000","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8380","last_scrub_stamp":"2024-05-09T07:39:44.430840+0000","last_deep_scrub":"27'8380","last_deep_scrub_stamp":"2024-05-09T07:39:44.430840+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.430840+0000","objects_scrubbed":6,"log_size":7380,"log_dups_size":0,"ondisk_log_size":7380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:50:04.135704+0000","scrub_duration":68,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971520,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8370,"num_write_kb":75026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3406,"num_omap_keys":6,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'9858","reported_seq":10338,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.345070+0000","last_change":"2024-05-09T07:39:56.345111+0000","last_active":"2024-05-09T07:39:56.345070+0000","last_peered":"2024-05-09T07:39:56.345070+0000","last_clean":"2024-05-09T07:39:56.345070+0000","last_became_active":"2024-05-09T07:13:49.507279+0000","last_became_peered":"2024-05-09T07:13:49.507279+0000","last_unstale":"2024-05-09T07:39:56.345070+0000","last_undegraded":"2024-05-09T07:39:56.345070+0000","last_fullsized":"2024-05-09T07:39:56.345070+0000","mapping_epoch":22,"log_start":"27'2476","ondisk_log_start":"27'2476","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9858","last_scrub_stamp":"2024-05-09T07:39:56.344972+0000","last_deep_scrub":"27'9858","last_deep_scrub_stamp":"2024-05-09T07:39:56.344972+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.344972+0000","objects_scrubbed":0,"log_size":7382,"log_dups_size":0,"ondisk_log_size":7382,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:32.670316+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":438,"num_read_kb":111252,"num_write":7195,"num_write_kb":963969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'2574","reported_seq":2620,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.256006+0000","last_change":"2024-05-09T07:39:44.256058+0000","last_active":"2024-05-09T07:39:44.256006+0000","last_peered":"2024-05-09T07:39:44.256006+0000","last_clean":"2024-05-09T07:39:44.256006+0000","last_became_active":"2024-05-09T07:13:45.471686+0000","last_became_peered":"2024-05-09T07:13:45.471686+0000","last_unstale":"2024-05-09T07:39:44.256006+0000","last_undegraded":"2024-05-09T07:39:44.256006+0000","last_fullsized":"2024-05-09T07:39:44.256006+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2574","last_scrub_stamp":"2024-05-09T07:39:44.255885+0000","last_deep_scrub":"27'2574","last_deep_scrub_stamp":"2024-05-09T07:39:44.255885+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.255885+0000","objects_scrubbed":0,"log_size":2574,"log_dups_size":0,"ondisk_log_size":2574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:50.387900+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'9200","reported_seq":9249,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.130589+0000","last_change":"2024-05-09T07:39:49.130620+0000","last_active":"2024-05-09T07:39:49.130589+0000","last_peered":"2024-05-09T07:39:49.130589+0000","last_clean":"2024-05-09T07:39:49.130589+0000","last_became_active":"2024-05-09T07:13:44.467836+0000","last_became_peered":"2024-05-09T07:13:44.467836+0000","last_unstale":"2024-05-09T07:39:49.130589+0000","last_undegraded":"2024-05-09T07:39:49.130589+0000","last_fullsized":"2024-05-09T07:39:49.130589+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9200","last_scrub_stamp":"2024-05-09T07:39:49.130519+0000","last_deep_scrub":"27'9200","last_deep_scrub_stamp":"2024-05-09T07:39:49.130519+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.130519+0000","objects_scrubbed":2,"log_size":7400,"log_dups_size":0,"ondisk_log_size":7400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:54.307815+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9200,"num_write_kb":63563,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'9948","reported_seq":10122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.353807+0000","last_change":"2024-05-09T07:39:54.353857+0000","last_active":"2024-05-09T07:39:54.353807+0000","last_peered":"2024-05-09T07:39:54.353807+0000","last_clean":"2024-05-09T07:39:54.353807+0000","last_became_active":"2024-05-09T07:13:49.501476+0000","last_became_peered":"2024-05-09T07:13:49.501476+0000","last_unstale":"2024-05-09T07:39:54.353807+0000","last_undegraded":"2024-05-09T07:39:54.353807+0000","last_fullsized":"2024-05-09T07:39:54.353807+0000","mapping_epoch":22,"log_start":"27'2616","ondisk_log_start":"27'2616","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9948","last_scrub_stamp":"2024-05-09T07:39:54.353678+0000","last_deep_scrub":"27'9948","last_deep_scrub_stamp":"2024-05-09T07:39:54.353678+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.353678+0000","objects_scrubbed":0,"log_size":7332,"log_dups_size":0,"ondisk_log_size":7332,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:30.970283+0000","scrub_duration":42,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":33528,"num_write":7177,"num_write_kb":934133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'2581","reported_seq":2629,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.343555+0000","last_change":"2024-05-09T07:39:53.343576+0000","last_active":"2024-05-09T07:39:53.343555+0000","last_peered":"2024-05-09T07:39:53.343555+0000","last_clean":"2024-05-09T07:39:53.343555+0000","last_became_active":"2024-05-09T07:13:45.478769+0000","last_became_peered":"2024-05-09T07:13:45.478769+0000","last_unstale":"2024-05-09T07:39:53.343555+0000","last_undegraded":"2024-05-09T07:39:53.343555+0000","last_fullsized":"2024-05-09T07:39:53.343555+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2581","last_scrub_stamp":"2024-05-09T07:39:53.343492+0000","last_deep_scrub":"27'2581","last_deep_scrub_stamp":"2024-05-09T07:39:53.343492+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.343492+0000","objects_scrubbed":0,"log_size":2581,"log_dups_size":0,"ondisk_log_size":2581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:50.650233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'7809","reported_seq":8757,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.810778+0000","last_change":"2024-05-09T07:39:44.810810+0000","last_active":"2024-05-09T07:39:44.810778+0000","last_peered":"2024-05-09T07:39:44.810778+0000","last_clean":"2024-05-09T07:39:44.810778+0000","last_became_active":"2024-05-09T07:13:44.471349+0000","last_became_peered":"2024-05-09T07:13:44.471349+0000","last_unstale":"2024-05-09T07:39:44.810778+0000","last_undegraded":"2024-05-09T07:39:44.810778+0000","last_fullsized":"2024-05-09T07:39:44.810778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7809","last_scrub_stamp":"2024-05-09T07:39:44.810716+0000","last_deep_scrub":"27'7809","last_deep_scrub_stamp":"2024-05-09T07:39:44.810716+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.810716+0000","objects_scrubbed":3,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:18:30.496241+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8607492,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":900,"num_read_kb":4675,"num_write":7792,"num_write_kb":62664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'9564","reported_seq":9949,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.301961+0000","last_change":"2024-05-09T07:39:55.301961+0000","last_active":"2024-05-09T07:39:55.301961+0000","last_peered":"2024-05-09T07:39:55.301961+0000","last_clean":"2024-05-09T07:39:55.301961+0000","last_became_active":"2024-05-09T07:13:49.502761+0000","last_became_peered":"2024-05-09T07:13:49.502761+0000","last_unstale":"2024-05-09T07:39:55.301961+0000","last_undegraded":"2024-05-09T07:39:55.301961+0000","last_fullsized":"2024-05-09T07:39:55.301961+0000","mapping_epoch":22,"log_start":"27'2181","ondisk_log_start":"27'2181","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9564","last_scrub_stamp":"2024-05-09T07:39:55.301777+0000","last_deep_scrub":"27'9564","last_deep_scrub_stamp":"2024-05-09T07:39:55.301777+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.301777+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:18.512745+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":343,"num_read_kb":87122,"num_write":6858,"num_write_kb":899446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'2524","reported_seq":2570,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.676783+0000","last_change":"2024-05-09T07:39:45.676804+0000","last_active":"2024-05-09T07:39:45.676783+0000","last_peered":"2024-05-09T07:39:45.676783+0000","last_clean":"2024-05-09T07:39:45.676783+0000","last_became_active":"2024-05-09T07:13:45.473490+0000","last_became_peered":"2024-05-09T07:13:45.473490+0000","last_unstale":"2024-05-09T07:39:45.676783+0000","last_undegraded":"2024-05-09T07:39:45.676783+0000","last_fullsized":"2024-05-09T07:39:45.676783+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2524","last_scrub_stamp":"2024-05-09T07:39:45.676715+0000","last_deep_scrub":"27'2524","last_deep_scrub_stamp":"2024-05-09T07:39:45.676715+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.676715+0000","objects_scrubbed":0,"log_size":2524,"log_dups_size":0,"ondisk_log_size":2524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:13.837357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7853","reported_seq":7902,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.268486+0000","last_change":"2024-05-09T07:39:50.268486+0000","last_active":"2024-05-09T07:39:50.268486+0000","last_peered":"2024-05-09T07:39:50.268486+0000","last_clean":"2024-05-09T07:39:50.268486+0000","last_became_active":"2024-05-09T07:13:44.475288+0000","last_became_peered":"2024-05-09T07:13:44.475288+0000","last_unstale":"2024-05-09T07:39:50.268486+0000","last_undegraded":"2024-05-09T07:39:50.268486+0000","last_fullsized":"2024-05-09T07:39:50.268486+0000","mapping_epoch":17,"log_start":"27'800","ondisk_log_start":"27'800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7853","last_scrub_stamp":"2024-05-09T07:39:50.268374+0000","last_deep_scrub":"27'7853","last_deep_scrub_stamp":"2024-05-09T07:39:50.268374+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.268374+0000","objects_scrubbed":1,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:55.445670+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7854,"num_write_kb":58481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'9413","reported_seq":9677,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.578326+0000","last_change":"2024-05-09T07:39:55.578351+0000","last_active":"2024-05-09T07:39:55.578326+0000","last_peered":"2024-05-09T07:39:55.578326+0000","last_clean":"2024-05-09T07:39:55.578326+0000","last_became_active":"2024-05-09T07:13:49.938640+0000","last_became_peered":"2024-05-09T07:13:49.938640+0000","last_unstale":"2024-05-09T07:39:55.578326+0000","last_undegraded":"2024-05-09T07:39:55.578326+0000","last_fullsized":"2024-05-09T07:39:55.578326+0000","mapping_epoch":22,"log_start":"27'2042","ondisk_log_start":"27'2042","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9413","last_scrub_stamp":"2024-05-09T07:39:55.578237+0000","last_deep_scrub":"27'9413","last_deep_scrub_stamp":"2024-05-09T07:39:55.578237+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.578237+0000","objects_scrubbed":0,"log_size":7371,"log_dups_size":0,"ondisk_log_size":7371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:51.820544+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":222,"num_read_kb":56388,"num_write":6719,"num_write_kb":1113426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'2642","reported_seq":2689,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.220909+0000","last_change":"2024-05-09T07:39:45.220931+0000","last_active":"2024-05-09T07:39:45.220909+0000","last_peered":"2024-05-09T07:39:45.220909+0000","last_clean":"2024-05-09T07:39:45.220909+0000","last_became_active":"2024-05-09T07:13:45.489223+0000","last_became_peered":"2024-05-09T07:13:45.489223+0000","last_unstale":"2024-05-09T07:39:45.220909+0000","last_undegraded":"2024-05-09T07:39:45.220909+0000","last_fullsized":"2024-05-09T07:39:45.220909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2642","last_scrub_stamp":"2024-05-09T07:39:45.220837+0000","last_deep_scrub":"27'2642","last_deep_scrub_stamp":"2024-05-09T07:39:45.220837+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.220837+0000","objects_scrubbed":0,"log_size":2642,"log_dups_size":0,"ondisk_log_size":2642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:43.566468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'4652","reported_seq":4701,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.108555+0000","last_change":"2024-05-09T07:39:48.108619+0000","last_active":"2024-05-09T07:39:48.108555+0000","last_peered":"2024-05-09T07:39:48.108555+0000","last_clean":"2024-05-09T07:39:48.108555+0000","last_became_active":"2024-05-09T07:13:44.467624+0000","last_became_peered":"2024-05-09T07:13:44.467624+0000","last_unstale":"2024-05-09T07:39:48.108555+0000","last_undegraded":"2024-05-09T07:39:48.108555+0000","last_fullsized":"2024-05-09T07:39:48.108555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4652","last_scrub_stamp":"2024-05-09T07:39:48.108460+0000","last_deep_scrub":"27'4652","last_deep_scrub_stamp":"2024-05-09T07:39:48.108460+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.108460+0000","objects_scrubbed":1,"log_size":4652,"log_dups_size":0,"ondisk_log_size":4652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:21:28.934163+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4655,"num_write_kb":39916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":207,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.417352+0000","last_change":"2024-05-09T07:39:45.417389+0000","last_active":"2024-05-09T07:39:45.417352+0000","last_peered":"2024-05-09T07:39:45.417352+0000","last_clean":"2024-05-09T07:39:45.417352+0000","last_became_active":"2024-05-09T07:13:34.337125+0000","last_became_peered":"2024-05-09T07:13:34.337125+0000","last_unstale":"2024-05-09T07:39:45.417352+0000","last_undegraded":"2024-05-09T07:39:45.417352+0000","last_fullsized":"2024-05-09T07:39:45.417352+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-05-09T07:39:45.417268+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-05-09T07:39:45.417268+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.417268+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:10:58.531645+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'8970","reported_seq":9599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.849579+0000","last_change":"2024-05-09T07:39:57.849622+0000","last_active":"2024-05-09T07:39:57.849579+0000","last_peered":"2024-05-09T07:39:57.849579+0000","last_clean":"2024-05-09T07:39:57.849579+0000","last_became_active":"2024-05-09T07:13:49.940781+0000","last_became_peered":"2024-05-09T07:13:49.940781+0000","last_unstale":"2024-05-09T07:39:57.849579+0000","last_undegraded":"2024-05-09T07:39:57.849579+0000","last_fullsized":"2024-05-09T07:39:57.849579+0000","mapping_epoch":22,"log_start":"27'1027","ondisk_log_start":"27'1027","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8970","last_scrub_stamp":"2024-05-09T07:39:57.849401+0000","last_deep_scrub":"27'8970","last_deep_scrub_stamp":"2024-05-09T07:39:57.849401+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.849401+0000","objects_scrubbed":0,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:08.614258+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":587,"num_read_kb":149098,"num_write":6333,"num_write_kb":900398,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'2565","reported_seq":2595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.709609+0000","last_change":"2024-05-09T07:39:49.709651+0000","last_active":"2024-05-09T07:39:49.709609+0000","last_peered":"2024-05-09T07:39:49.709609+0000","last_clean":"2024-05-09T07:39:49.709609+0000","last_became_active":"2024-05-09T07:14:26.591447+0000","last_became_peered":"2024-05-09T07:14:26.591447+0000","last_unstale":"2024-05-09T07:39:49.709609+0000","last_undegraded":"2024-05-09T07:39:49.709609+0000","last_fullsized":"2024-05-09T07:39:49.709609+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2565","last_scrub_stamp":"2024-05-09T07:39:49.709531+0000","last_deep_scrub":"27'2565","last_deep_scrub_stamp":"2024-05-09T07:39:49.709531+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.709531+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:35:00.704431+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"27'7877","reported_seq":7924,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.759260+0000","last_change":"2024-05-09T07:39:45.759396+0000","last_active":"2024-05-09T07:39:45.759260+0000","last_peered":"2024-05-09T07:39:45.759260+0000","last_clean":"2024-05-09T07:39:45.759260+0000","last_became_active":"2024-05-09T07:13:44.471383+0000","last_became_peered":"2024-05-09T07:13:44.471383+0000","last_unstale":"2024-05-09T07:39:45.759260+0000","last_undegraded":"2024-05-09T07:39:45.759260+0000","last_fullsized":"2024-05-09T07:39:45.759260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7877","last_scrub_stamp":"2024-05-09T07:39:45.759163+0000","last_deep_scrub":"27'7877","last_deep_scrub_stamp":"2024-05-09T07:39:45.759163+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.759163+0000","objects_scrubbed":2,"log_size":7877,"log_dups_size":0,"ondisk_log_size":7877,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:02:34.122563+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7879,"num_write_kb":58494,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'9733","reported_seq":10224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.820745+0000","last_change":"2024-05-09T07:39:56.820745+0000","last_active":"2024-05-09T07:39:56.820745+0000","last_peered":"2024-05-09T07:39:56.820745+0000","last_clean":"2024-05-09T07:39:56.820745+0000","last_became_active":"2024-05-09T07:13:49.938833+0000","last_became_peered":"2024-05-09T07:13:49.938833+0000","last_unstale":"2024-05-09T07:39:56.820745+0000","last_undegraded":"2024-05-09T07:39:56.820745+0000","last_fullsized":"2024-05-09T07:39:56.820745+0000","mapping_epoch":22,"log_start":"27'1834","ondisk_log_start":"27'1834","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9733","last_scrub_stamp":"2024-05-09T07:39:56.820610+0000","last_deep_scrub":"27'9733","last_deep_scrub_stamp":"2024-05-09T07:39:56.820610+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.820610+0000","objects_scrubbed":0,"log_size":7899,"log_dups_size":0,"ondisk_log_size":7899,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:22.442335+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":449,"num_read_kb":114046,"num_write":7019,"num_write_kb":962272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'2589","reported_seq":2635,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.022806+0000","last_change":"2024-05-09T07:39:51.022834+0000","last_active":"2024-05-09T07:39:51.022806+0000","last_peered":"2024-05-09T07:39:51.022806+0000","last_clean":"2024-05-09T07:39:51.022806+0000","last_became_active":"2024-05-09T07:13:45.482996+0000","last_became_peered":"2024-05-09T07:13:45.482996+0000","last_unstale":"2024-05-09T07:39:51.022806+0000","last_undegraded":"2024-05-09T07:39:51.022806+0000","last_fullsized":"2024-05-09T07:39:51.022806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2589","last_scrub_stamp":"2024-05-09T07:39:51.022717+0000","last_deep_scrub":"27'2589","last_deep_scrub_stamp":"2024-05-09T07:39:51.022717+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.022717+0000","objects_scrubbed":0,"log_size":2589,"log_dups_size":0,"ondisk_log_size":2589,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:09.129494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7547","reported_seq":7595,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.351048+0000","last_change":"2024-05-09T07:39:45.351076+0000","last_active":"2024-05-09T07:39:45.351048+0000","last_peered":"2024-05-09T07:39:45.351048+0000","last_clean":"2024-05-09T07:39:45.351048+0000","last_became_active":"2024-05-09T07:13:44.470201+0000","last_became_peered":"2024-05-09T07:13:44.470201+0000","last_unstale":"2024-05-09T07:39:45.351048+0000","last_undegraded":"2024-05-09T07:39:45.351048+0000","last_fullsized":"2024-05-09T07:39:45.351048+0000","mapping_epoch":17,"log_start":"27'200","ondisk_log_start":"27'200","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7547","last_scrub_stamp":"2024-05-09T07:39:45.350982+0000","last_deep_scrub":"27'7547","last_deep_scrub_stamp":"2024-05-09T07:39:45.350982+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.350982+0000","objects_scrubbed":3,"log_size":7347,"log_dups_size":0,"ondisk_log_size":7347,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:45.621741+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388678,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":7546,"num_write_kb":58281,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'9829","reported_seq":10121,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.624925+0000","last_change":"2024-05-09T07:39:54.624950+0000","last_active":"2024-05-09T07:39:54.624925+0000","last_peered":"2024-05-09T07:39:54.624925+0000","last_clean":"2024-05-09T07:39:54.624925+0000","last_became_active":"2024-05-09T07:13:49.506376+0000","last_became_peered":"2024-05-09T07:13:49.506376+0000","last_unstale":"2024-05-09T07:39:54.624925+0000","last_undegraded":"2024-05-09T07:39:54.624925+0000","last_fullsized":"2024-05-09T07:39:54.624925+0000","mapping_epoch":22,"log_start":"27'2499","ondisk_log_start":"27'2499","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9829","last_scrub_stamp":"2024-05-09T07:39:54.624831+0000","last_deep_scrub":"27'9829","last_deep_scrub_stamp":"2024-05-09T07:39:54.624831+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.624831+0000","objects_scrubbed":0,"log_size":7330,"log_dups_size":0,"ondisk_log_size":7330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:17.835798+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":251,"num_read_kb":63754,"num_write":7056,"num_write_kb":893799,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'2456","reported_seq":2501,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:59.252073+0000","last_change":"2024-05-09T07:39:59.252117+0000","last_active":"2024-05-09T07:39:59.252073+0000","last_peered":"2024-05-09T07:39:59.252073+0000","last_clean":"2024-05-09T07:39:59.252073+0000","last_became_active":"2024-05-09T07:13:45.497374+0000","last_became_peered":"2024-05-09T07:13:45.497374+0000","last_unstale":"2024-05-09T07:39:59.252073+0000","last_undegraded":"2024-05-09T07:39:59.252073+0000","last_fullsized":"2024-05-09T07:39:59.252073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2456","last_scrub_stamp":"2024-05-09T07:39:59.251965+0000","last_deep_scrub":"27'2456","last_deep_scrub_stamp":"2024-05-09T07:39:59.251965+0000","last_clean_scrub_stamp":"2024-05-09T07:39:59.251965+0000","objects_scrubbed":0,"log_size":2456,"log_dups_size":0,"ondisk_log_size":2456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:02.422533+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8870","reported_seq":8917,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.871046+0000","last_change":"2024-05-09T07:39:43.871134+0000","last_active":"2024-05-09T07:39:43.871046+0000","last_peered":"2024-05-09T07:39:43.871046+0000","last_clean":"2024-05-09T07:39:43.871046+0000","last_became_active":"2024-05-09T07:13:44.471282+0000","last_became_peered":"2024-05-09T07:13:44.471282+0000","last_unstale":"2024-05-09T07:39:43.871046+0000","last_undegraded":"2024-05-09T07:39:43.871046+0000","last_fullsized":"2024-05-09T07:39:43.871046+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8870","last_scrub_stamp":"2024-05-09T07:39:43.870938+0000","last_deep_scrub":"27'8870","last_deep_scrub_stamp":"2024-05-09T07:39:43.870938+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.870938+0000","objects_scrubbed":7,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:19:08.222552+0000","scrub_duration":91,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25165824,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":8854,"num_write_kb":79483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'9365","reported_seq":10390,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:59.007672+0000","last_change":"2024-05-09T07:39:59.007715+0000","last_active":"2024-05-09T07:39:59.007672+0000","last_peered":"2024-05-09T07:39:59.007672+0000","last_clean":"2024-05-09T07:39:59.007672+0000","last_became_active":"2024-05-09T07:13:49.940898+0000","last_became_peered":"2024-05-09T07:13:49.940898+0000","last_unstale":"2024-05-09T07:39:59.007672+0000","last_undegraded":"2024-05-09T07:39:59.007672+0000","last_fullsized":"2024-05-09T07:39:59.007672+0000","mapping_epoch":22,"log_start":"27'1982","ondisk_log_start":"27'1982","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9365","last_scrub_stamp":"2024-05-09T07:39:59.007562+0000","last_deep_scrub":"27'9365","last_deep_scrub_stamp":"2024-05-09T07:39:59.007562+0000","last_clean_scrub_stamp":"2024-05-09T07:39:59.007562+0000","objects_scrubbed":0,"log_size":7383,"log_dups_size":0,"ondisk_log_size":7383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:29.859023+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":983,"num_read_kb":249682,"num_write":6568,"num_write_kb":1037240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'2557","reported_seq":2602,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.275849+0000","last_change":"2024-05-09T07:39:53.275875+0000","last_active":"2024-05-09T07:39:53.275849+0000","last_peered":"2024-05-09T07:39:53.275849+0000","last_clean":"2024-05-09T07:39:53.275849+0000","last_became_active":"2024-05-09T07:13:45.486836+0000","last_became_peered":"2024-05-09T07:13:45.486836+0000","last_unstale":"2024-05-09T07:39:53.275849+0000","last_undegraded":"2024-05-09T07:39:53.275849+0000","last_fullsized":"2024-05-09T07:39:53.275849+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2557","last_scrub_stamp":"2024-05-09T07:39:53.275754+0000","last_deep_scrub":"27'2557","last_deep_scrub_stamp":"2024-05-09T07:39:53.275754+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.275754+0000","objects_scrubbed":0,"log_size":2557,"log_dups_size":0,"ondisk_log_size":2557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:39.108105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"27'5467","reported_seq":5514,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.298874+0000","last_change":"2024-05-09T07:39:43.298937+0000","last_active":"2024-05-09T07:39:43.298874+0000","last_peered":"2024-05-09T07:39:43.298874+0000","last_clean":"2024-05-09T07:39:43.298874+0000","last_became_active":"2024-05-09T07:13:44.469720+0000","last_became_peered":"2024-05-09T07:13:44.469720+0000","last_unstale":"2024-05-09T07:39:43.298874+0000","last_undegraded":"2024-05-09T07:39:43.298874+0000","last_fullsized":"2024-05-09T07:39:43.298874+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5467","last_scrub_stamp":"2024-05-09T07:39:43.298807+0000","last_deep_scrub":"27'5467","last_deep_scrub_stamp":"2024-05-09T07:39:43.298807+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.298807+0000","objects_scrubbed":2,"log_size":5467,"log_dups_size":0,"ondisk_log_size":5467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:09.732787+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194326,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5458,"num_write_kb":44489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'9305","reported_seq":9874,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.305576+0000","last_change":"2024-05-09T07:39:51.305576+0000","last_active":"2024-05-09T07:39:51.305576+0000","last_peered":"2024-05-09T07:39:51.305576+0000","last_clean":"2024-05-09T07:39:51.305576+0000","last_became_active":"2024-05-09T07:13:49.505443+0000","last_became_peered":"2024-05-09T07:13:49.505443+0000","last_unstale":"2024-05-09T07:39:51.305576+0000","last_undegraded":"2024-05-09T07:39:51.305576+0000","last_fullsized":"2024-05-09T07:39:51.305576+0000","mapping_epoch":22,"log_start":"27'2103","ondisk_log_start":"27'2103","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9305","last_scrub_stamp":"2024-05-09T07:39:51.305425+0000","last_deep_scrub":"27'9305","last_deep_scrub_stamp":"2024-05-09T07:39:51.305425+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.305425+0000","objects_scrubbed":0,"log_size":7202,"log_dups_size":0,"ondisk_log_size":7202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:09.412202+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":527,"num_read_kb":133858,"num_write":6613,"num_write_kb":868619,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'2624","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:58.263568+0000","last_change":"2024-05-09T07:39:58.263613+0000","last_active":"2024-05-09T07:39:58.263568+0000","last_peered":"2024-05-09T07:39:58.263568+0000","last_clean":"2024-05-09T07:39:58.263568+0000","last_became_active":"2024-05-09T07:13:45.497498+0000","last_became_peered":"2024-05-09T07:13:45.497498+0000","last_unstale":"2024-05-09T07:39:58.263568+0000","last_undegraded":"2024-05-09T07:39:58.263568+0000","last_fullsized":"2024-05-09T07:39:58.263568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2624","last_scrub_stamp":"2024-05-09T07:39:58.263467+0000","last_deep_scrub":"27'2624","last_deep_scrub_stamp":"2024-05-09T07:39:58.263467+0000","last_clean_scrub_stamp":"2024-05-09T07:39:58.263467+0000","objects_scrubbed":0,"log_size":2624,"log_dups_size":0,"ondisk_log_size":2624,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:34.588812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'3474","reported_seq":3521,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.779308+0000","last_change":"2024-05-09T07:39:42.779351+0000","last_active":"2024-05-09T07:39:42.779308+0000","last_peered":"2024-05-09T07:39:42.779308+0000","last_clean":"2024-05-09T07:39:42.779308+0000","last_became_active":"2024-05-09T07:13:44.468332+0000","last_became_peered":"2024-05-09T07:13:44.468332+0000","last_unstale":"2024-05-09T07:39:42.779308+0000","last_undegraded":"2024-05-09T07:39:42.779308+0000","last_fullsized":"2024-05-09T07:39:42.779308+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3474","last_scrub_stamp":"2024-05-09T07:39:42.779246+0000","last_deep_scrub":"27'3474","last_deep_scrub_stamp":"2024-05-09T07:39:42.779246+0000","last_clean_scrub_stamp":"2024-05-09T07:39:42.779246+0000","objects_scrubbed":6,"log_size":3474,"log_dups_size":0,"ondisk_log_size":3474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:47.753557+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":3482,"num_write_kb":39113,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'9477","reported_seq":10061,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.603567+0000","last_change":"2024-05-09T07:39:53.603611+0000","last_active":"2024-05-09T07:39:53.603567+0000","last_peered":"2024-05-09T07:39:53.603567+0000","last_clean":"2024-05-09T07:39:53.603567+0000","last_became_active":"2024-05-09T07:13:49.505749+0000","last_became_peered":"2024-05-09T07:13:49.505749+0000","last_unstale":"2024-05-09T07:39:53.603567+0000","last_undegraded":"2024-05-09T07:39:53.603567+0000","last_fullsized":"2024-05-09T07:39:53.603567+0000","mapping_epoch":22,"log_start":"27'2114","ondisk_log_start":"27'2114","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9477","last_scrub_stamp":"2024-05-09T07:39:53.603458+0000","last_deep_scrub":"27'9477","last_deep_scrub_stamp":"2024-05-09T07:39:53.603458+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.603458+0000","objects_scrubbed":0,"log_size":7363,"log_dups_size":0,"ondisk_log_size":7363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:30.220063+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":544,"num_read_kb":138176,"num_write":6755,"num_write_kb":1181456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'2498","reported_seq":2544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.853727+0000","last_change":"2024-05-09T07:39:48.853781+0000","last_active":"2024-05-09T07:39:48.853727+0000","last_peered":"2024-05-09T07:39:48.853727+0000","last_clean":"2024-05-09T07:39:48.853727+0000","last_became_active":"2024-05-09T07:13:45.483821+0000","last_became_peered":"2024-05-09T07:13:45.483821+0000","last_unstale":"2024-05-09T07:39:48.853727+0000","last_undegraded":"2024-05-09T07:39:48.853727+0000","last_fullsized":"2024-05-09T07:39:48.853727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2498","last_scrub_stamp":"2024-05-09T07:39:48.853626+0000","last_deep_scrub":"27'2498","last_deep_scrub_stamp":"2024-05-09T07:39:48.853626+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.853626+0000","objects_scrubbed":0,"log_size":2498,"log_dups_size":0,"ondisk_log_size":2498,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:49.124385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'4971","reported_seq":5019,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.976174+0000","last_change":"2024-05-09T07:39:44.976243+0000","last_active":"2024-05-09T07:39:44.976174+0000","last_peered":"2024-05-09T07:39:44.976174+0000","last_clean":"2024-05-09T07:39:44.976174+0000","last_became_active":"2024-05-09T07:13:44.465309+0000","last_became_peered":"2024-05-09T07:13:44.465309+0000","last_unstale":"2024-05-09T07:39:44.976174+0000","last_undegraded":"2024-05-09T07:39:44.976174+0000","last_fullsized":"2024-05-09T07:39:44.976174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4971","last_scrub_stamp":"2024-05-09T07:39:44.976095+0000","last_deep_scrub":"27'4971","last_deep_scrub_stamp":"2024-05-09T07:39:44.976095+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.976095+0000","objects_scrubbed":3,"log_size":4971,"log_dups_size":0,"ondisk_log_size":4971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:11:17.477682+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12582912,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":0,"num_write":4979,"num_write_kb":44143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'9318","reported_seq":9630,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:03.296759+0000","last_change":"2024-05-09T07:40:03.296784+0000","last_active":"2024-05-09T07:40:03.296759+0000","last_peered":"2024-05-09T07:40:03.296759+0000","last_clean":"2024-05-09T07:40:03.296759+0000","last_became_active":"2024-05-09T07:13:49.941574+0000","last_became_peered":"2024-05-09T07:13:49.941574+0000","last_unstale":"2024-05-09T07:40:03.296759+0000","last_undegraded":"2024-05-09T07:40:03.296759+0000","last_fullsized":"2024-05-09T07:40:03.296759+0000","mapping_epoch":22,"log_start":"27'3038","ondisk_log_start":"27'3038","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9318","last_scrub_stamp":"2024-05-09T07:40:03.296672+0000","last_deep_scrub":"27'9318","last_deep_scrub_stamp":"2024-05-09T07:40:03.296672+0000","last_clean_scrub_stamp":"2024-05-09T07:40:03.296672+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:01:43.825748+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":272,"num_read_kb":69088,"num_write":6517,"num_write_kb":924175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'2603","reported_seq":2648,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:00.196618+0000","last_change":"2024-05-09T07:40:00.196646+0000","last_active":"2024-05-09T07:40:00.196618+0000","last_peered":"2024-05-09T07:40:00.196618+0000","last_clean":"2024-05-09T07:40:00.196618+0000","last_became_active":"2024-05-09T07:13:45.497927+0000","last_became_peered":"2024-05-09T07:13:45.497927+0000","last_unstale":"2024-05-09T07:40:00.196618+0000","last_undegraded":"2024-05-09T07:40:00.196618+0000","last_fullsized":"2024-05-09T07:40:00.196618+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2603","last_scrub_stamp":"2024-05-09T07:40:00.196534+0000","last_deep_scrub":"27'2603","last_deep_scrub_stamp":"2024-05-09T07:40:00.196534+0000","last_clean_scrub_stamp":"2024-05-09T07:40:00.196534+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:31.047995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'2653","reported_seq":2701,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.119973+0000","last_change":"2024-05-09T07:39:46.120005+0000","last_active":"2024-05-09T07:39:46.119973+0000","last_peered":"2024-05-09T07:39:46.119973+0000","last_clean":"2024-05-09T07:39:46.119973+0000","last_became_active":"2024-05-09T07:13:44.460333+0000","last_became_peered":"2024-05-09T07:13:44.460333+0000","last_unstale":"2024-05-09T07:39:46.119973+0000","last_undegraded":"2024-05-09T07:39:46.119973+0000","last_fullsized":"2024-05-09T07:39:46.119973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2653","last_scrub_stamp":"2024-05-09T07:39:46.119886+0000","last_deep_scrub":"27'2653","last_deep_scrub_stamp":"2024-05-09T07:39:46.119886+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.119886+0000","objects_scrubbed":2,"log_size":2653,"log_dups_size":0,"ondisk_log_size":2653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:23.572755+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2652,"num_write_kb":30266,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'9224","reported_seq":9462,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:00.299709+0000","last_change":"2024-05-09T07:40:00.299757+0000","last_active":"2024-05-09T07:40:00.299709+0000","last_peered":"2024-05-09T07:40:00.299709+0000","last_clean":"2024-05-09T07:40:00.299709+0000","last_became_active":"2024-05-09T07:13:49.938643+0000","last_became_peered":"2024-05-09T07:13:49.938643+0000","last_unstale":"2024-05-09T07:40:00.299709+0000","last_undegraded":"2024-05-09T07:40:00.299709+0000","last_fullsized":"2024-05-09T07:40:00.299709+0000","mapping_epoch":22,"log_start":"27'2939","ondisk_log_start":"27'2939","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9224","last_scrub_stamp":"2024-05-09T07:40:00.299587+0000","last_deep_scrub":"27'9224","last_deep_scrub_stamp":"2024-05-09T07:40:00.299587+0000","last_clean_scrub_stamp":"2024-05-09T07:40:00.299587+0000","objects_scrubbed":0,"log_size":6285,"log_dups_size":0,"ondisk_log_size":6285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:21:38.648282+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":197,"num_read_kb":50038,"num_write":6664,"num_write_kb":910804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'2562","reported_seq":2607,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.559321+0000","last_change":"2024-05-09T07:39:51.559344+0000","last_active":"2024-05-09T07:39:51.559321+0000","last_peered":"2024-05-09T07:39:51.559321+0000","last_clean":"2024-05-09T07:39:51.559321+0000","last_became_active":"2024-05-09T07:13:45.492373+0000","last_became_peered":"2024-05-09T07:13:45.492373+0000","last_unstale":"2024-05-09T07:39:51.559321+0000","last_undegraded":"2024-05-09T07:39:51.559321+0000","last_fullsized":"2024-05-09T07:39:51.559321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2562","last_scrub_stamp":"2024-05-09T07:39:51.559252+0000","last_deep_scrub":"27'2562","last_deep_scrub_stamp":"2024-05-09T07:39:51.559252+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.559252+0000","objects_scrubbed":0,"log_size":2562,"log_dups_size":0,"ondisk_log_size":2562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:07.899467+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'7004","reported_seq":7051,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:42.717054+0000","last_change":"2024-05-09T07:39:42.717102+0000","last_active":"2024-05-09T07:39:42.717054+0000","last_peered":"2024-05-09T07:39:42.717054+0000","last_clean":"2024-05-09T07:39:42.717054+0000","last_became_active":"2024-05-09T07:13:44.466124+0000","last_became_peered":"2024-05-09T07:13:44.466124+0000","last_unstale":"2024-05-09T07:39:42.717054+0000","last_undegraded":"2024-05-09T07:39:42.717054+0000","last_fullsized":"2024-05-09T07:39:42.717054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7004","last_scrub_stamp":"2024-05-09T07:39:42.716991+0000","last_deep_scrub":"27'7004","last_deep_scrub_stamp":"2024-05-09T07:39:42.716991+0000","last_clean_scrub_stamp":"2024-05-09T07:39:42.716991+0000","objects_scrubbed":7,"log_size":7004,"log_dups_size":0,"ondisk_log_size":7004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:16:48.849006+0000","scrub_duration":56,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20971554,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6989,"num_write_kb":57820,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'9732","reported_seq":10009,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:59.958509+0000","last_change":"2024-05-09T07:39:59.958536+0000","last_active":"2024-05-09T07:39:59.958509+0000","last_peered":"2024-05-09T07:39:59.958509+0000","last_clean":"2024-05-09T07:39:59.958509+0000","last_became_active":"2024-05-09T07:13:49.942774+0000","last_became_peered":"2024-05-09T07:13:49.942774+0000","last_unstale":"2024-05-09T07:39:59.958509+0000","last_undegraded":"2024-05-09T07:39:59.958509+0000","last_fullsized":"2024-05-09T07:39:59.958509+0000","mapping_epoch":22,"log_start":"27'2359","ondisk_log_start":"27'2359","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9732","last_scrub_stamp":"2024-05-09T07:39:59.958394+0000","last_deep_scrub":"27'9732","last_deep_scrub_stamp":"2024-05-09T07:39:59.958394+0000","last_clean_scrub_stamp":"2024-05-09T07:39:59.958394+0000","objects_scrubbed":0,"log_size":7373,"log_dups_size":0,"ondisk_log_size":7373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:44:31.840013+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":237,"num_read_kb":60198,"num_write":7045,"num_write_kb":1050276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'2506","reported_seq":2550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.427897+0000","last_change":"2024-05-09T07:39:44.427929+0000","last_active":"2024-05-09T07:39:44.427897+0000","last_peered":"2024-05-09T07:39:44.427897+0000","last_clean":"2024-05-09T07:39:44.427897+0000","last_became_active":"2024-05-09T07:13:45.486607+0000","last_became_peered":"2024-05-09T07:13:45.486607+0000","last_unstale":"2024-05-09T07:39:44.427897+0000","last_undegraded":"2024-05-09T07:39:44.427897+0000","last_fullsized":"2024-05-09T07:39:44.427897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2506","last_scrub_stamp":"2024-05-09T07:39:44.427778+0000","last_deep_scrub":"27'2506","last_deep_scrub_stamp":"2024-05-09T07:39:44.427778+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.427778+0000","objects_scrubbed":0,"log_size":2506,"log_dups_size":0,"ondisk_log_size":2506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:55.077769+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'10547","reported_seq":11222,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.946840+0000","last_change":"2024-05-09T07:39:45.946881+0000","last_active":"2024-05-09T07:39:45.946840+0000","last_peered":"2024-05-09T07:39:45.946840+0000","last_clean":"2024-05-09T07:39:45.946840+0000","last_became_active":"2024-05-09T07:13:44.466082+0000","last_became_peered":"2024-05-09T07:13:44.466082+0000","last_unstale":"2024-05-09T07:39:45.946840+0000","last_undegraded":"2024-05-09T07:39:45.946840+0000","last_fullsized":"2024-05-09T07:39:45.946840+0000","mapping_epoch":17,"log_start":"27'2600","ondisk_log_start":"27'2600","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10547","last_scrub_stamp":"2024-05-09T07:39:45.946752+0000","last_deep_scrub":"27'10547","last_deep_scrub_stamp":"2024-05-09T07:39:45.946752+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.946752+0000","objects_scrubbed":5,"log_size":7947,"log_dups_size":0,"ondisk_log_size":7947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:56.596743+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8389280,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":627,"num_read_kb":4520,"num_write":10536,"num_write_kb":76795,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'9196","reported_seq":9449,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:01.002183+0000","last_change":"2024-05-09T07:40:01.002183+0000","last_active":"2024-05-09T07:40:01.002183+0000","last_peered":"2024-05-09T07:40:01.002183+0000","last_clean":"2024-05-09T07:40:01.002183+0000","last_became_active":"2024-05-09T07:14:26.590721+0000","last_became_peered":"2024-05-09T07:14:26.590721+0000","last_unstale":"2024-05-09T07:40:01.002183+0000","last_undegraded":"2024-05-09T07:40:01.002183+0000","last_fullsized":"2024-05-09T07:40:01.002183+0000","mapping_epoch":26,"log_start":"27'1828","ondisk_log_start":"27'1828","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9196","last_scrub_stamp":"2024-05-09T07:40:01.002068+0000","last_deep_scrub":"27'9196","last_deep_scrub_stamp":"2024-05-09T07:40:01.002068+0000","last_clean_scrub_stamp":"2024-05-09T07:40:01.002068+0000","objects_scrubbed":0,"log_size":7368,"log_dups_size":0,"ondisk_log_size":7368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:50:03.491439+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":221,"num_read_kb":56134,"num_write":6509,"num_write_kb":978701,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2,0],"acting":[5,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"27'2577","reported_seq":2622,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.287304+0000","last_change":"2024-05-09T07:39:47.287327+0000","last_active":"2024-05-09T07:39:47.287304+0000","last_peered":"2024-05-09T07:39:47.287304+0000","last_clean":"2024-05-09T07:39:47.287304+0000","last_became_active":"2024-05-09T07:13:45.494034+0000","last_became_peered":"2024-05-09T07:13:45.494034+0000","last_unstale":"2024-05-09T07:39:47.287304+0000","last_undegraded":"2024-05-09T07:39:47.287304+0000","last_fullsized":"2024-05-09T07:39:47.287304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2577","last_scrub_stamp":"2024-05-09T07:39:47.287227+0000","last_deep_scrub":"27'2577","last_deep_scrub_stamp":"2024-05-09T07:39:47.287227+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.287227+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:46.335272+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'6180","reported_seq":6228,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.313543+0000","last_change":"2024-05-09T07:39:46.313543+0000","last_active":"2024-05-09T07:39:46.313543+0000","last_peered":"2024-05-09T07:39:46.313543+0000","last_clean":"2024-05-09T07:39:46.313543+0000","last_became_active":"2024-05-09T07:13:44.470903+0000","last_became_peered":"2024-05-09T07:13:44.470903+0000","last_unstale":"2024-05-09T07:39:46.313543+0000","last_undegraded":"2024-05-09T07:39:46.313543+0000","last_fullsized":"2024-05-09T07:39:46.313543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6180","last_scrub_stamp":"2024-05-09T07:39:46.313400+0000","last_deep_scrub":"27'6180","last_deep_scrub_stamp":"2024-05-09T07:39:46.313400+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.313400+0000","objects_scrubbed":2,"log_size":6180,"log_dups_size":0,"ondisk_log_size":6180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:45.361444+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":6185,"num_write_kb":41396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'9171","reported_seq":9524,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.250909+0000","last_change":"2024-05-09T07:39:57.250938+0000","last_active":"2024-05-09T07:39:57.250909+0000","last_peered":"2024-05-09T07:39:57.250909+0000","last_clean":"2024-05-09T07:39:57.250909+0000","last_became_active":"2024-05-09T07:14:26.595936+0000","last_became_peered":"2024-05-09T07:14:26.595936+0000","last_unstale":"2024-05-09T07:39:57.250909+0000","last_undegraded":"2024-05-09T07:39:57.250909+0000","last_fullsized":"2024-05-09T07:39:57.250909+0000","mapping_epoch":26,"log_start":"27'2010","ondisk_log_start":"27'2010","created":22,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9171","last_scrub_stamp":"2024-05-09T07:39:57.250803+0000","last_deep_scrub":"27'9171","last_deep_scrub_stamp":"2024-05-09T07:39:57.250803+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.250803+0000","objects_scrubbed":0,"log_size":7161,"log_dups_size":0,"ondisk_log_size":7161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:53.493110+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":305,"num_read_kb":77470,"num_write":6446,"num_write_kb":989255,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,2],"acting":[4,5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'2611","reported_seq":2655,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:41.667993+0000","last_change":"2024-05-09T07:39:41.668036+0000","last_active":"2024-05-09T07:39:41.667993+0000","last_peered":"2024-05-09T07:39:41.667993+0000","last_clean":"2024-05-09T07:39:41.667993+0000","last_became_active":"2024-05-09T07:13:45.488876+0000","last_became_peered":"2024-05-09T07:13:45.488876+0000","last_unstale":"2024-05-09T07:39:41.667993+0000","last_undegraded":"2024-05-09T07:39:41.667993+0000","last_fullsized":"2024-05-09T07:39:41.667993+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2611","last_scrub_stamp":"2024-05-09T07:39:41.667866+0000","last_deep_scrub":"27'2611","last_deep_scrub_stamp":"2024-05-09T07:39:41.667866+0000","last_clean_scrub_stamp":"2024-05-09T07:39:41.667866+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:48:52.317856+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'5743","reported_seq":5790,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.085294+0000","last_change":"2024-05-09T07:39:50.085331+0000","last_active":"2024-05-09T07:39:50.085294+0000","last_peered":"2024-05-09T07:39:50.085294+0000","last_clean":"2024-05-09T07:39:50.085294+0000","last_became_active":"2024-05-09T07:13:44.470181+0000","last_became_peered":"2024-05-09T07:13:44.470181+0000","last_unstale":"2024-05-09T07:39:50.085294+0000","last_undegraded":"2024-05-09T07:39:50.085294+0000","last_fullsized":"2024-05-09T07:39:50.085294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5743","last_scrub_stamp":"2024-05-09T07:39:50.085191+0000","last_deep_scrub":"27'5743","last_deep_scrub_stamp":"2024-05-09T07:39:50.085191+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.085191+0000","objects_scrubbed":2,"log_size":5743,"log_dups_size":0,"ondisk_log_size":5743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:40.947993+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":5742,"num_write_kb":52939,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'9121","reported_seq":9497,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.329311+0000","last_change":"2024-05-09T07:39:57.329372+0000","last_active":"2024-05-09T07:39:57.329311+0000","last_peered":"2024-05-09T07:39:57.329311+0000","last_clean":"2024-05-09T07:39:57.329311+0000","last_became_active":"2024-05-09T07:13:49.503794+0000","last_became_peered":"2024-05-09T07:13:49.503794+0000","last_unstale":"2024-05-09T07:39:57.329311+0000","last_undegraded":"2024-05-09T07:39:57.329311+0000","last_fullsized":"2024-05-09T07:39:57.329311+0000","mapping_epoch":22,"log_start":"27'2805","ondisk_log_start":"27'2805","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9121","last_scrub_stamp":"2024-05-09T07:39:57.329183+0000","last_deep_scrub":"27'9121","last_deep_scrub_stamp":"2024-05-09T07:39:57.329183+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.329183+0000","objects_scrubbed":0,"log_size":6316,"log_dups_size":0,"ondisk_log_size":6316,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:02:56.341286+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":335,"num_read_kb":85090,"num_write":6449,"num_write_kb":967440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'2587","reported_seq":2632,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.985851+0000","last_change":"2024-05-09T07:39:54.985876+0000","last_active":"2024-05-09T07:39:54.985851+0000","last_peered":"2024-05-09T07:39:54.985851+0000","last_clean":"2024-05-09T07:39:54.985851+0000","last_became_active":"2024-05-09T07:13:45.492497+0000","last_became_peered":"2024-05-09T07:13:45.492497+0000","last_unstale":"2024-05-09T07:39:54.985851+0000","last_undegraded":"2024-05-09T07:39:54.985851+0000","last_fullsized":"2024-05-09T07:39:54.985851+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2587","last_scrub_stamp":"2024-05-09T07:39:54.985767+0000","last_deep_scrub":"27'2587","last_deep_scrub_stamp":"2024-05-09T07:39:54.985767+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.985767+0000","objects_scrubbed":0,"log_size":2587,"log_dups_size":0,"ondisk_log_size":2587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:59:31.602372+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'5946","reported_seq":6859,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.892149+0000","last_change":"2024-05-09T07:39:46.892175+0000","last_active":"2024-05-09T07:39:46.892149+0000","last_peered":"2024-05-09T07:39:46.892149+0000","last_clean":"2024-05-09T07:39:46.892149+0000","last_became_active":"2024-05-09T07:13:44.471087+0000","last_became_peered":"2024-05-09T07:13:44.471087+0000","last_unstale":"2024-05-09T07:39:46.892149+0000","last_undegraded":"2024-05-09T07:39:46.892149+0000","last_fullsized":"2024-05-09T07:39:46.892149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5946","last_scrub_stamp":"2024-05-09T07:39:46.892064+0000","last_deep_scrub":"27'5946","last_deep_scrub_stamp":"2024-05-09T07:39:46.892064+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.892064+0000","objects_scrubbed":0,"log_size":5946,"log_dups_size":0,"ondisk_log_size":5946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:00.006440+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":867,"num_read_kb":4659,"num_write":5947,"num_write_kb":49048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'9603","reported_seq":10000,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:40:01.340723+0000","last_change":"2024-05-09T07:40:01.340748+0000","last_active":"2024-05-09T07:40:01.340723+0000","last_peered":"2024-05-09T07:40:01.340723+0000","last_clean":"2024-05-09T07:40:01.340723+0000","last_became_active":"2024-05-09T07:13:49.940135+0000","last_became_peered":"2024-05-09T07:13:49.940135+0000","last_unstale":"2024-05-09T07:40:01.340723+0000","last_undegraded":"2024-05-09T07:40:01.340723+0000","last_fullsized":"2024-05-09T07:40:01.340723+0000","mapping_epoch":22,"log_start":"27'3323","ondisk_log_start":"27'3323","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9603","last_scrub_stamp":"2024-05-09T07:40:01.340643+0000","last_deep_scrub":"27'9603","last_deep_scrub_stamp":"2024-05-09T07:40:01.340643+0000","last_clean_scrub_stamp":"2024-05-09T07:40:01.340643+0000","objects_scrubbed":0,"log_size":6280,"log_dups_size":0,"ondisk_log_size":6280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:06:09.781177+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":356,"num_read_kb":90424,"num_write":6822,"num_write_kb":1059940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'9164","reported_seq":9401,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.557104+0000","last_change":"2024-05-09T07:39:57.557104+0000","last_active":"2024-05-09T07:39:57.557104+0000","last_peered":"2024-05-09T07:39:57.557104+0000","last_clean":"2024-05-09T07:39:57.557104+0000","last_became_active":"2024-05-09T07:13:49.940238+0000","last_became_peered":"2024-05-09T07:13:49.940238+0000","last_unstale":"2024-05-09T07:39:57.557104+0000","last_undegraded":"2024-05-09T07:39:57.557104+0000","last_fullsized":"2024-05-09T07:39:57.557104+0000","mapping_epoch":22,"log_start":"27'1813","ondisk_log_start":"27'1813","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9164","last_scrub_stamp":"2024-05-09T07:39:57.556982+0000","last_deep_scrub":"27'9164","last_deep_scrub_stamp":"2024-05-09T07:39:57.556982+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.556982+0000","objects_scrubbed":0,"log_size":7351,"log_dups_size":0,"ondisk_log_size":7351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:29:28.408443+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":194,"num_read_kb":49276,"num_write":6561,"num_write_kb":912358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'2535","reported_seq":2580,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.380110+0000","last_change":"2024-05-09T07:39:51.380151+0000","last_active":"2024-05-09T07:39:51.380110+0000","last_peered":"2024-05-09T07:39:51.380110+0000","last_clean":"2024-05-09T07:39:51.380110+0000","last_became_active":"2024-05-09T07:13:45.477432+0000","last_became_peered":"2024-05-09T07:13:45.477432+0000","last_unstale":"2024-05-09T07:39:51.380110+0000","last_undegraded":"2024-05-09T07:39:51.380110+0000","last_fullsized":"2024-05-09T07:39:51.380110+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2535","last_scrub_stamp":"2024-05-09T07:39:51.380024+0000","last_deep_scrub":"27'2535","last_deep_scrub_stamp":"2024-05-09T07:39:51.380024+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.380024+0000","objects_scrubbed":0,"log_size":2535,"log_dups_size":0,"ondisk_log_size":2535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:43:51.708529+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'9072","reported_seq":9119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.101955+0000","last_change":"2024-05-09T07:39:47.101986+0000","last_active":"2024-05-09T07:39:47.101955+0000","last_peered":"2024-05-09T07:39:47.101955+0000","last_clean":"2024-05-09T07:39:47.101955+0000","last_became_active":"2024-05-09T07:13:44.466193+0000","last_became_peered":"2024-05-09T07:13:44.466193+0000","last_unstale":"2024-05-09T07:39:47.101955+0000","last_undegraded":"2024-05-09T07:39:47.101955+0000","last_fullsized":"2024-05-09T07:39:47.101955+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9072","last_scrub_stamp":"2024-05-09T07:39:47.101861+0000","last_deep_scrub":"27'9072","last_deep_scrub_stamp":"2024-05-09T07:39:47.101861+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.101861+0000","objects_scrubbed":1,"log_size":7372,"log_dups_size":0,"ondisk_log_size":7372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:22.953252+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9068,"num_write_kb":67566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'8844","reported_seq":10039,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.768345+0000","last_change":"2024-05-09T07:39:55.768399+0000","last_active":"2024-05-09T07:39:55.768345+0000","last_peered":"2024-05-09T07:39:55.768345+0000","last_clean":"2024-05-09T07:39:55.768345+0000","last_became_active":"2024-05-09T07:13:49.506820+0000","last_became_peered":"2024-05-09T07:13:49.506820+0000","last_unstale":"2024-05-09T07:39:55.768345+0000","last_undegraded":"2024-05-09T07:39:55.768345+0000","last_fullsized":"2024-05-09T07:39:55.768345+0000","mapping_epoch":22,"log_start":"27'900","ondisk_log_start":"27'900","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8844","last_scrub_stamp":"2024-05-09T07:39:55.768221+0000","last_deep_scrub":"27'8844","last_deep_scrub_stamp":"2024-05-09T07:39:55.768221+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.768221+0000","objects_scrubbed":0,"log_size":7944,"log_dups_size":0,"ondisk_log_size":7944,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:12.108436+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1153,"num_read_kb":292862,"num_write":6082,"num_write_kb":877058,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'2633","reported_seq":2678,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.253693+0000","last_change":"2024-05-09T07:39:52.253732+0000","last_active":"2024-05-09T07:39:52.253693+0000","last_peered":"2024-05-09T07:39:52.253693+0000","last_clean":"2024-05-09T07:39:52.253693+0000","last_became_active":"2024-05-09T07:13:45.478655+0000","last_became_peered":"2024-05-09T07:13:45.478655+0000","last_unstale":"2024-05-09T07:39:52.253693+0000","last_undegraded":"2024-05-09T07:39:52.253693+0000","last_fullsized":"2024-05-09T07:39:52.253693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2633","last_scrub_stamp":"2024-05-09T07:39:52.253615+0000","last_deep_scrub":"27'2633","last_deep_scrub_stamp":"2024-05-09T07:39:52.253615+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.253615+0000","objects_scrubbed":0,"log_size":2633,"log_dups_size":0,"ondisk_log_size":2633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:03.018472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'5358","reported_seq":5405,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.297604+0000","last_change":"2024-05-09T07:39:50.297633+0000","last_active":"2024-05-09T07:39:50.297604+0000","last_peered":"2024-05-09T07:39:50.297604+0000","last_clean":"2024-05-09T07:39:50.297604+0000","last_became_active":"2024-05-09T07:13:44.469552+0000","last_became_peered":"2024-05-09T07:13:44.469552+0000","last_unstale":"2024-05-09T07:39:50.297604+0000","last_undegraded":"2024-05-09T07:39:50.297604+0000","last_fullsized":"2024-05-09T07:39:50.297604+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5358","last_scrub_stamp":"2024-05-09T07:39:50.297534+0000","last_deep_scrub":"27'5358","last_deep_scrub_stamp":"2024-05-09T07:39:50.297534+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.297534+0000","objects_scrubbed":2,"log_size":5358,"log_dups_size":0,"ondisk_log_size":5358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:06.637749+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8388608,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":5356,"num_write_kb":44481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'9698","reported_seq":9745,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.997220+0000","last_change":"2024-05-09T07:39:43.997249+0000","last_active":"2024-05-09T07:39:43.997220+0000","last_peered":"2024-05-09T07:39:43.997220+0000","last_clean":"2024-05-09T07:39:43.997220+0000","last_became_active":"2024-05-09T07:13:44.464316+0000","last_became_peered":"2024-05-09T07:13:44.464316+0000","last_unstale":"2024-05-09T07:39:43.997220+0000","last_undegraded":"2024-05-09T07:39:43.997220+0000","last_fullsized":"2024-05-09T07:39:43.997220+0000","mapping_epoch":17,"log_start":"27'1800","ondisk_log_start":"27'1800","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9698","last_scrub_stamp":"2024-05-09T07:39:43.997141+0000","last_deep_scrub":"27'9698","last_deep_scrub_stamp":"2024-05-09T07:39:43.997141+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.997141+0000","objects_scrubbed":1,"log_size":7898,"log_dups_size":0,"ondisk_log_size":7898,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:24.963666+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":9694,"num_write_kb":72019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2561","reported_seq":2605,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.709885+0000","last_change":"2024-05-09T07:39:46.709907+0000","last_active":"2024-05-09T07:39:46.709885+0000","last_peered":"2024-05-09T07:39:46.709885+0000","last_clean":"2024-05-09T07:39:46.709885+0000","last_became_active":"2024-05-09T07:13:45.473362+0000","last_became_peered":"2024-05-09T07:13:45.473362+0000","last_unstale":"2024-05-09T07:39:46.709885+0000","last_undegraded":"2024-05-09T07:39:46.709885+0000","last_fullsized":"2024-05-09T07:39:46.709885+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2561","last_scrub_stamp":"2024-05-09T07:39:46.709806+0000","last_deep_scrub":"27'2561","last_deep_scrub_stamp":"2024-05-09T07:39:46.709806+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.709806+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:45.055437+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'9401","reported_seq":9465,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.703699+0000","last_change":"2024-05-09T07:39:48.703725+0000","last_active":"2024-05-09T07:39:48.703699+0000","last_peered":"2024-05-09T07:39:48.703699+0000","last_clean":"2024-05-09T07:39:48.703699+0000","last_became_active":"2024-05-09T07:13:49.494428+0000","last_became_peered":"2024-05-09T07:13:49.494428+0000","last_unstale":"2024-05-09T07:39:48.703699+0000","last_undegraded":"2024-05-09T07:39:48.703699+0000","last_fullsized":"2024-05-09T07:39:48.703699+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9401","last_scrub_stamp":"2024-05-09T07:39:48.703611+0000","last_deep_scrub":"27'9401","last_deep_scrub_stamp":"2024-05-09T07:39:48.703611+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.703611+0000","objects_scrubbed":0,"log_size":9401,"log_dups_size":0,"ondisk_log_size":9401,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:46:29.670136+0000","scrub_duration":48,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":24,"num_read_kb":6096,"num_write":6603,"num_write_kb":983207,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'8029","reported_seq":8076,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.228151+0000","last_change":"2024-05-09T07:39:49.228172+0000","last_active":"2024-05-09T07:39:49.228151+0000","last_peered":"2024-05-09T07:39:49.228151+0000","last_clean":"2024-05-09T07:39:49.228151+0000","last_became_active":"2024-05-09T07:13:44.465567+0000","last_became_peered":"2024-05-09T07:13:44.465567+0000","last_unstale":"2024-05-09T07:39:49.228151+0000","last_undegraded":"2024-05-09T07:39:49.228151+0000","last_fullsized":"2024-05-09T07:39:49.228151+0000","mapping_epoch":17,"log_start":"27'1700","ondisk_log_start":"27'1700","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8029","last_scrub_stamp":"2024-05-09T07:39:49.228089+0000","last_deep_scrub":"27'8029","last_deep_scrub_stamp":"2024-05-09T07:39:49.228089+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.228089+0000","objects_scrubbed":0,"log_size":6329,"log_dups_size":0,"ondisk_log_size":6329,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:35.060440+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":8031,"num_write_kb":66635,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'2603","reported_seq":2647,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:45.246682+0000","last_change":"2024-05-09T07:39:45.246722+0000","last_active":"2024-05-09T07:39:45.246682+0000","last_peered":"2024-05-09T07:39:45.246682+0000","last_clean":"2024-05-09T07:39:45.246682+0000","last_became_active":"2024-05-09T07:13:45.476326+0000","last_became_peered":"2024-05-09T07:13:45.476326+0000","last_unstale":"2024-05-09T07:39:45.246682+0000","last_undegraded":"2024-05-09T07:39:45.246682+0000","last_fullsized":"2024-05-09T07:39:45.246682+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2603","last_scrub_stamp":"2024-05-09T07:39:45.246606+0000","last_deep_scrub":"27'2603","last_deep_scrub_stamp":"2024-05-09T07:39:45.246606+0000","last_clean_scrub_stamp":"2024-05-09T07:39:45.246606+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:23:22.699475+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'9830","reported_seq":9991,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:58.386218+0000","last_change":"2024-05-09T07:39:58.386260+0000","last_active":"2024-05-09T07:39:58.386218+0000","last_peered":"2024-05-09T07:39:58.386218+0000","last_clean":"2024-05-09T07:39:58.386218+0000","last_became_active":"2024-05-09T07:13:49.504216+0000","last_became_peered":"2024-05-09T07:13:49.504216+0000","last_unstale":"2024-05-09T07:39:58.386218+0000","last_undegraded":"2024-05-09T07:39:58.386218+0000","last_fullsized":"2024-05-09T07:39:58.386218+0000","mapping_epoch":22,"log_start":"27'3543","ondisk_log_start":"27'3543","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9830","last_scrub_stamp":"2024-05-09T07:39:58.386093+0000","last_deep_scrub":"27'9830","last_deep_scrub_stamp":"2024-05-09T07:39:58.386093+0000","last_clean_scrub_stamp":"2024-05-09T07:39:58.386093+0000","objects_scrubbed":0,"log_size":6287,"log_dups_size":0,"ondisk_log_size":6287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:26:11.093394+0000","scrub_duration":64,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":30734,"num_write":7017,"num_write_kb":1030226,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'2510","reported_seq":2557,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.807824+0000","last_change":"2024-05-09T07:39:51.807849+0000","last_active":"2024-05-09T07:39:51.807824+0000","last_peered":"2024-05-09T07:39:51.807824+0000","last_clean":"2024-05-09T07:39:51.807824+0000","last_became_active":"2024-05-09T07:13:45.489100+0000","last_became_peered":"2024-05-09T07:13:45.489100+0000","last_unstale":"2024-05-09T07:39:51.807824+0000","last_undegraded":"2024-05-09T07:39:51.807824+0000","last_fullsized":"2024-05-09T07:39:51.807824+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2510","last_scrub_stamp":"2024-05-09T07:39:51.807731+0000","last_deep_scrub":"27'2510","last_deep_scrub_stamp":"2024-05-09T07:39:51.807731+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.807731+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:56.985027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'2625","reported_seq":2671,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:46.687922+0000","last_change":"2024-05-09T07:39:46.687962+0000","last_active":"2024-05-09T07:39:46.687922+0000","last_peered":"2024-05-09T07:39:46.687922+0000","last_clean":"2024-05-09T07:39:46.687922+0000","last_became_active":"2024-05-09T07:13:45.486401+0000","last_became_peered":"2024-05-09T07:13:45.486401+0000","last_unstale":"2024-05-09T07:39:46.687922+0000","last_undegraded":"2024-05-09T07:39:46.687922+0000","last_fullsized":"2024-05-09T07:39:46.687922+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2625","last_scrub_stamp":"2024-05-09T07:39:46.687817+0000","last_deep_scrub":"27'2625","last_deep_scrub_stamp":"2024-05-09T07:39:46.687817+0000","last_clean_scrub_stamp":"2024-05-09T07:39:46.687817+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T19:09:28.909565+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'2528","reported_seq":2575,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.008243+0000","last_change":"2024-05-09T07:39:54.008268+0000","last_active":"2024-05-09T07:39:54.008243+0000","last_peered":"2024-05-09T07:39:54.008243+0000","last_clean":"2024-05-09T07:39:54.008243+0000","last_became_active":"2024-05-09T07:13:45.490135+0000","last_became_peered":"2024-05-09T07:13:45.490135+0000","last_unstale":"2024-05-09T07:39:54.008243+0000","last_undegraded":"2024-05-09T07:39:54.008243+0000","last_fullsized":"2024-05-09T07:39:54.008243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2528","last_scrub_stamp":"2024-05-09T07:39:54.008158+0000","last_deep_scrub":"27'2528","last_deep_scrub_stamp":"2024-05-09T07:39:54.008158+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.008158+0000","objects_scrubbed":0,"log_size":2528,"log_dups_size":0,"ondisk_log_size":2528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:51.314898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'2612","reported_seq":2658,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.662490+0000","last_change":"2024-05-09T07:39:47.662514+0000","last_active":"2024-05-09T07:39:47.662490+0000","last_peered":"2024-05-09T07:39:47.662490+0000","last_clean":"2024-05-09T07:39:47.662490+0000","last_became_active":"2024-05-09T07:13:45.475586+0000","last_became_peered":"2024-05-09T07:13:45.475586+0000","last_unstale":"2024-05-09T07:39:47.662490+0000","last_undegraded":"2024-05-09T07:39:47.662490+0000","last_fullsized":"2024-05-09T07:39:47.662490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2612","last_scrub_stamp":"2024-05-09T07:39:47.662413+0000","last_deep_scrub":"27'2612","last_deep_scrub_stamp":"2024-05-09T07:39:47.662413+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.662413+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:00:52.839709+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'2606","reported_seq":2653,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.271140+0000","last_change":"2024-05-09T07:39:54.271164+0000","last_active":"2024-05-09T07:39:54.271140+0000","last_peered":"2024-05-09T07:39:54.271140+0000","last_clean":"2024-05-09T07:39:54.271140+0000","last_became_active":"2024-05-09T07:13:45.483178+0000","last_became_peered":"2024-05-09T07:13:45.483178+0000","last_unstale":"2024-05-09T07:39:54.271140+0000","last_undegraded":"2024-05-09T07:39:54.271140+0000","last_fullsized":"2024-05-09T07:39:54.271140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2606","last_scrub_stamp":"2024-05-09T07:39:54.271071+0000","last_deep_scrub":"27'2606","last_deep_scrub_stamp":"2024-05-09T07:39:54.271071+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.271071+0000","objects_scrubbed":0,"log_size":2606,"log_dups_size":0,"ondisk_log_size":2606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:42:30.596415+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'2485","reported_seq":2532,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:50.352606+0000","last_change":"2024-05-09T07:39:50.352606+0000","last_active":"2024-05-09T07:39:50.352606+0000","last_peered":"2024-05-09T07:39:50.352606+0000","last_clean":"2024-05-09T07:39:50.352606+0000","last_became_active":"2024-05-09T07:13:45.475447+0000","last_became_peered":"2024-05-09T07:13:45.475447+0000","last_unstale":"2024-05-09T07:39:50.352606+0000","last_undegraded":"2024-05-09T07:39:50.352606+0000","last_fullsized":"2024-05-09T07:39:50.352606+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2485","last_scrub_stamp":"2024-05-09T07:39:50.352482+0000","last_deep_scrub":"27'2485","last_deep_scrub_stamp":"2024-05-09T07:39:50.352482+0000","last_clean_scrub_stamp":"2024-05-09T07:39:50.352482+0000","objects_scrubbed":0,"log_size":2485,"log_dups_size":0,"ondisk_log_size":2485,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:16:47.994271+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'2622","reported_seq":2669,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.782107+0000","last_change":"2024-05-09T07:39:54.782134+0000","last_active":"2024-05-09T07:39:54.782107+0000","last_peered":"2024-05-09T07:39:54.782107+0000","last_clean":"2024-05-09T07:39:54.782107+0000","last_became_active":"2024-05-09T07:13:45.493017+0000","last_became_peered":"2024-05-09T07:13:45.493017+0000","last_unstale":"2024-05-09T07:39:54.782107+0000","last_undegraded":"2024-05-09T07:39:54.782107+0000","last_fullsized":"2024-05-09T07:39:54.782107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2622","last_scrub_stamp":"2024-05-09T07:39:54.782014+0000","last_deep_scrub":"27'2622","last_deep_scrub_stamp":"2024-05-09T07:39:54.782014+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.782014+0000","objects_scrubbed":0,"log_size":2622,"log_dups_size":0,"ondisk_log_size":2622,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:40.614366+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'2594","reported_seq":2641,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.282537+0000","last_change":"2024-05-09T07:39:55.282577+0000","last_active":"2024-05-09T07:39:55.282537+0000","last_peered":"2024-05-09T07:39:55.282537+0000","last_clean":"2024-05-09T07:39:55.282537+0000","last_became_active":"2024-05-09T07:13:45.483172+0000","last_became_peered":"2024-05-09T07:13:45.483172+0000","last_unstale":"2024-05-09T07:39:55.282537+0000","last_undegraded":"2024-05-09T07:39:55.282537+0000","last_fullsized":"2024-05-09T07:39:55.282537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2594","last_scrub_stamp":"2024-05-09T07:39:55.282417+0000","last_deep_scrub":"27'2594","last_deep_scrub_stamp":"2024-05-09T07:39:55.282417+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.282417+0000","objects_scrubbed":0,"log_size":2594,"log_dups_size":0,"ondisk_log_size":2594,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:58.452985+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'2576","reported_seq":2606,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:51.702895+0000","last_change":"2024-05-09T07:39:51.702917+0000","last_active":"2024-05-09T07:39:51.702895+0000","last_peered":"2024-05-09T07:39:51.702895+0000","last_clean":"2024-05-09T07:39:51.702895+0000","last_became_active":"2024-05-09T07:14:26.646058+0000","last_became_peered":"2024-05-09T07:14:26.646058+0000","last_unstale":"2024-05-09T07:39:51.702895+0000","last_undegraded":"2024-05-09T07:39:51.702895+0000","last_fullsized":"2024-05-09T07:39:51.702895+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2576","last_scrub_stamp":"2024-05-09T07:39:51.702818+0000","last_deep_scrub":"27'2576","last_deep_scrub_stamp":"2024-05-09T07:39:51.702818+0000","last_clean_scrub_stamp":"2024-05-09T07:39:51.702818+0000","objects_scrubbed":0,"log_size":2576,"log_dups_size":0,"ondisk_log_size":2576,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:11:04.817195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'2541","reported_seq":2587,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.230572+0000","last_change":"2024-05-09T07:39:47.230610+0000","last_active":"2024-05-09T07:39:47.230572+0000","last_peered":"2024-05-09T07:39:47.230572+0000","last_clean":"2024-05-09T07:39:47.230572+0000","last_became_active":"2024-05-09T07:13:45.486336+0000","last_became_peered":"2024-05-09T07:13:45.486336+0000","last_unstale":"2024-05-09T07:39:47.230572+0000","last_undegraded":"2024-05-09T07:39:47.230572+0000","last_fullsized":"2024-05-09T07:39:47.230572+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2541","last_scrub_stamp":"2024-05-09T07:39:47.230473+0000","last_deep_scrub":"27'2541","last_deep_scrub_stamp":"2024-05-09T07:39:47.230473+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.230473+0000","objects_scrubbed":0,"log_size":2541,"log_dups_size":0,"ondisk_log_size":2541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:16.973851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"27'2609","reported_seq":2655,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.981280+0000","last_change":"2024-05-09T07:39:56.981303+0000","last_active":"2024-05-09T07:39:56.981280+0000","last_peered":"2024-05-09T07:39:56.981280+0000","last_clean":"2024-05-09T07:39:56.981280+0000","last_became_active":"2024-05-09T07:13:45.493498+0000","last_became_peered":"2024-05-09T07:13:45.493498+0000","last_unstale":"2024-05-09T07:39:56.981280+0000","last_undegraded":"2024-05-09T07:39:56.981280+0000","last_fullsized":"2024-05-09T07:39:56.981280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2609","last_scrub_stamp":"2024-05-09T07:39:56.981197+0000","last_deep_scrub":"27'2609","last_deep_scrub_stamp":"2024-05-09T07:39:56.981197+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.981197+0000","objects_scrubbed":0,"log_size":2609,"log_dups_size":0,"ondisk_log_size":2609,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:47:53.223504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2621","reported_seq":2666,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.561840+0000","last_change":"2024-05-09T07:39:52.561884+0000","last_active":"2024-05-09T07:39:52.561840+0000","last_peered":"2024-05-09T07:39:52.561840+0000","last_clean":"2024-05-09T07:39:52.561840+0000","last_became_active":"2024-05-09T07:13:45.493647+0000","last_became_peered":"2024-05-09T07:13:45.493647+0000","last_unstale":"2024-05-09T07:39:52.561840+0000","last_undegraded":"2024-05-09T07:39:52.561840+0000","last_fullsized":"2024-05-09T07:39:52.561840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2621","last_scrub_stamp":"2024-05-09T07:39:52.561754+0000","last_deep_scrub":"27'2621","last_deep_scrub_stamp":"2024-05-09T07:39:52.561754+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.561754+0000","objects_scrubbed":0,"log_size":2621,"log_dups_size":0,"ondisk_log_size":2621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:29:49.868495+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'2611","reported_seq":2657,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.824213+0000","last_change":"2024-05-09T07:39:52.824245+0000","last_active":"2024-05-09T07:39:52.824213+0000","last_peered":"2024-05-09T07:39:52.824213+0000","last_clean":"2024-05-09T07:39:52.824213+0000","last_became_active":"2024-05-09T07:13:45.488904+0000","last_became_peered":"2024-05-09T07:13:45.488904+0000","last_unstale":"2024-05-09T07:39:52.824213+0000","last_undegraded":"2024-05-09T07:39:52.824213+0000","last_fullsized":"2024-05-09T07:39:52.824213+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2611","last_scrub_stamp":"2024-05-09T07:39:52.824140+0000","last_deep_scrub":"27'2611","last_deep_scrub_stamp":"2024-05-09T07:39:52.824140+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.824140+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:51:22.567518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'2615","reported_seq":2660,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:54.292428+0000","last_change":"2024-05-09T07:39:54.292451+0000","last_active":"2024-05-09T07:39:54.292428+0000","last_peered":"2024-05-09T07:39:54.292428+0000","last_clean":"2024-05-09T07:39:54.292428+0000","last_became_active":"2024-05-09T07:13:45.490950+0000","last_became_peered":"2024-05-09T07:13:45.490950+0000","last_unstale":"2024-05-09T07:39:54.292428+0000","last_undegraded":"2024-05-09T07:39:54.292428+0000","last_fullsized":"2024-05-09T07:39:54.292428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2615","last_scrub_stamp":"2024-05-09T07:39:54.292348+0000","last_deep_scrub":"27'2615","last_deep_scrub_stamp":"2024-05-09T07:39:54.292348+0000","last_clean_scrub_stamp":"2024-05-09T07:39:54.292348+0000","objects_scrubbed":0,"log_size":2615,"log_dups_size":0,"ondisk_log_size":2615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:10.632563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'2554","reported_seq":2599,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:47.689317+0000","last_change":"2024-05-09T07:39:47.689358+0000","last_active":"2024-05-09T07:39:47.689317+0000","last_peered":"2024-05-09T07:39:47.689317+0000","last_clean":"2024-05-09T07:39:47.689317+0000","last_became_active":"2024-05-09T07:13:45.488137+0000","last_became_peered":"2024-05-09T07:13:45.488137+0000","last_unstale":"2024-05-09T07:39:47.689317+0000","last_undegraded":"2024-05-09T07:39:47.689317+0000","last_fullsized":"2024-05-09T07:39:47.689317+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2554","last_scrub_stamp":"2024-05-09T07:39:47.689226+0000","last_deep_scrub":"27'2554","last_deep_scrub_stamp":"2024-05-09T07:39:47.689226+0000","last_clean_scrub_stamp":"2024-05-09T07:39:47.689226+0000","objects_scrubbed":0,"log_size":2554,"log_dups_size":0,"ondisk_log_size":2554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:28.357856+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2625","reported_seq":2671,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:58.017424+0000","last_change":"2024-05-09T07:39:58.017448+0000","last_active":"2024-05-09T07:39:58.017424+0000","last_peered":"2024-05-09T07:39:58.017424+0000","last_clean":"2024-05-09T07:39:58.017424+0000","last_became_active":"2024-05-09T07:13:45.496072+0000","last_became_peered":"2024-05-09T07:13:45.496072+0000","last_unstale":"2024-05-09T07:39:58.017424+0000","last_undegraded":"2024-05-09T07:39:58.017424+0000","last_fullsized":"2024-05-09T07:39:58.017424+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2625","last_scrub_stamp":"2024-05-09T07:39:58.017310+0000","last_deep_scrub":"27'2625","last_deep_scrub_stamp":"2024-05-09T07:39:58.017310+0000","last_clean_scrub_stamp":"2024-05-09T07:39:58.017310+0000","objects_scrubbed":0,"log_size":2625,"log_dups_size":0,"ondisk_log_size":2625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:28:01.187878+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'2558","reported_seq":2604,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.092661+0000","last_change":"2024-05-09T07:39:44.092687+0000","last_active":"2024-05-09T07:39:44.092661+0000","last_peered":"2024-05-09T07:39:44.092661+0000","last_clean":"2024-05-09T07:39:44.092661+0000","last_became_active":"2024-05-09T07:13:45.489337+0000","last_became_peered":"2024-05-09T07:13:45.489337+0000","last_unstale":"2024-05-09T07:39:44.092661+0000","last_undegraded":"2024-05-09T07:39:44.092661+0000","last_fullsized":"2024-05-09T07:39:44.092661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2558","last_scrub_stamp":"2024-05-09T07:39:44.092580+0000","last_deep_scrub":"27'2558","last_deep_scrub_stamp":"2024-05-09T07:39:44.092580+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.092580+0000","objects_scrubbed":0,"log_size":2558,"log_dups_size":0,"ondisk_log_size":2558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:26:10.526561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'2678","reported_seq":2724,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.349461+0000","last_change":"2024-05-09T07:39:48.349500+0000","last_active":"2024-05-09T07:39:48.349461+0000","last_peered":"2024-05-09T07:39:48.349461+0000","last_clean":"2024-05-09T07:39:48.349461+0000","last_became_active":"2024-05-09T07:13:45.472654+0000","last_became_peered":"2024-05-09T07:13:45.472654+0000","last_unstale":"2024-05-09T07:39:48.349461+0000","last_undegraded":"2024-05-09T07:39:48.349461+0000","last_fullsized":"2024-05-09T07:39:48.349461+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2678","last_scrub_stamp":"2024-05-09T07:39:48.349373+0000","last_deep_scrub":"27'2678","last_deep_scrub_stamp":"2024-05-09T07:39:48.349373+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.349373+0000","objects_scrubbed":0,"log_size":2678,"log_dups_size":0,"ondisk_log_size":2678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:28:29.340354+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'2529","reported_seq":2574,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.630361+0000","last_change":"2024-05-09T07:39:48.630402+0000","last_active":"2024-05-09T07:39:48.630361+0000","last_peered":"2024-05-09T07:39:48.630361+0000","last_clean":"2024-05-09T07:39:48.630361+0000","last_became_active":"2024-05-09T07:13:45.476564+0000","last_became_peered":"2024-05-09T07:13:45.476564+0000","last_unstale":"2024-05-09T07:39:48.630361+0000","last_undegraded":"2024-05-09T07:39:48.630361+0000","last_fullsized":"2024-05-09T07:39:48.630361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2529","last_scrub_stamp":"2024-05-09T07:39:48.630274+0000","last_deep_scrub":"27'2529","last_deep_scrub_stamp":"2024-05-09T07:39:48.630274+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.630274+0000","objects_scrubbed":0,"log_size":2529,"log_dups_size":0,"ondisk_log_size":2529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:39.493076+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'2521","reported_seq":2567,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.217179+0000","last_change":"2024-05-09T07:39:44.217179+0000","last_active":"2024-05-09T07:39:44.217179+0000","last_peered":"2024-05-09T07:39:44.217179+0000","last_clean":"2024-05-09T07:39:44.217179+0000","last_became_active":"2024-05-09T07:13:45.490646+0000","last_became_peered":"2024-05-09T07:13:45.490646+0000","last_unstale":"2024-05-09T07:39:44.217179+0000","last_undegraded":"2024-05-09T07:39:44.217179+0000","last_fullsized":"2024-05-09T07:39:44.217179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2521","last_scrub_stamp":"2024-05-09T07:39:44.216997+0000","last_deep_scrub":"27'2521","last_deep_scrub_stamp":"2024-05-09T07:39:44.216997+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.216997+0000","objects_scrubbed":0,"log_size":2521,"log_dups_size":0,"ondisk_log_size":2521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:18:43.265042+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'2490","reported_seq":2535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:48.335382+0000","last_change":"2024-05-09T07:39:48.335382+0000","last_active":"2024-05-09T07:39:48.335382+0000","last_peered":"2024-05-09T07:39:48.335382+0000","last_clean":"2024-05-09T07:39:48.335382+0000","last_became_active":"2024-05-09T07:13:45.496116+0000","last_became_peered":"2024-05-09T07:13:45.496116+0000","last_unstale":"2024-05-09T07:39:48.335382+0000","last_undegraded":"2024-05-09T07:39:48.335382+0000","last_fullsized":"2024-05-09T07:39:48.335382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2490","last_scrub_stamp":"2024-05-09T07:39:48.335273+0000","last_deep_scrub":"27'2490","last_deep_scrub_stamp":"2024-05-09T07:39:48.335273+0000","last_clean_scrub_stamp":"2024-05-09T07:39:48.335273+0000","objects_scrubbed":0,"log_size":2490,"log_dups_size":0,"ondisk_log_size":2490,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T18:09:46.680904+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'2514","reported_seq":2559,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:53.819127+0000","last_change":"2024-05-09T07:39:53.819156+0000","last_active":"2024-05-09T07:39:53.819127+0000","last_peered":"2024-05-09T07:39:53.819127+0000","last_clean":"2024-05-09T07:39:53.819127+0000","last_became_active":"2024-05-09T07:13:45.491517+0000","last_became_peered":"2024-05-09T07:13:45.491517+0000","last_unstale":"2024-05-09T07:39:53.819127+0000","last_undegraded":"2024-05-09T07:39:53.819127+0000","last_fullsized":"2024-05-09T07:39:53.819127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2514","last_scrub_stamp":"2024-05-09T07:39:53.819024+0000","last_deep_scrub":"27'2514","last_deep_scrub_stamp":"2024-05-09T07:39:53.819024+0000","last_clean_scrub_stamp":"2024-05-09T07:39:53.819024+0000","objects_scrubbed":0,"log_size":2514,"log_dups_size":0,"ondisk_log_size":2514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:11.925801+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'2640","reported_seq":2685,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.899789+0000","last_change":"2024-05-09T07:39:49.899812+0000","last_active":"2024-05-09T07:39:49.899789+0000","last_peered":"2024-05-09T07:39:49.899789+0000","last_clean":"2024-05-09T07:39:49.899789+0000","last_became_active":"2024-05-09T07:13:45.486205+0000","last_became_peered":"2024-05-09T07:13:45.486205+0000","last_unstale":"2024-05-09T07:39:49.899789+0000","last_undegraded":"2024-05-09T07:39:49.899789+0000","last_fullsized":"2024-05-09T07:39:49.899789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2640","last_scrub_stamp":"2024-05-09T07:39:49.899700+0000","last_deep_scrub":"27'2640","last_deep_scrub_stamp":"2024-05-09T07:39:49.899700+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.899700+0000","objects_scrubbed":0,"log_size":2640,"log_dups_size":0,"ondisk_log_size":2640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:13:18.060341+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'2612","reported_seq":2656,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:56.307290+0000","last_change":"2024-05-09T07:39:56.307324+0000","last_active":"2024-05-09T07:39:56.307290+0000","last_peered":"2024-05-09T07:39:56.307290+0000","last_clean":"2024-05-09T07:39:56.307290+0000","last_became_active":"2024-05-09T07:13:45.491750+0000","last_became_peered":"2024-05-09T07:13:45.491750+0000","last_unstale":"2024-05-09T07:39:56.307290+0000","last_undegraded":"2024-05-09T07:39:56.307290+0000","last_fullsized":"2024-05-09T07:39:56.307290+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2612","last_scrub_stamp":"2024-05-09T07:39:56.307203+0000","last_deep_scrub":"27'2612","last_deep_scrub_stamp":"2024-05-09T07:39:56.307203+0000","last_clean_scrub_stamp":"2024-05-09T07:39:56.307203+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:54:07.072060+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'2548","reported_seq":2593,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:43.201784+0000","last_change":"2024-05-09T07:39:43.201825+0000","last_active":"2024-05-09T07:39:43.201784+0000","last_peered":"2024-05-09T07:39:43.201784+0000","last_clean":"2024-05-09T07:39:43.201784+0000","last_became_active":"2024-05-09T07:13:45.489444+0000","last_became_peered":"2024-05-09T07:13:45.489444+0000","last_unstale":"2024-05-09T07:39:43.201784+0000","last_undegraded":"2024-05-09T07:39:43.201784+0000","last_fullsized":"2024-05-09T07:39:43.201784+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2548","last_scrub_stamp":"2024-05-09T07:39:43.201670+0000","last_deep_scrub":"27'2548","last_deep_scrub_stamp":"2024-05-09T07:39:43.201670+0000","last_clean_scrub_stamp":"2024-05-09T07:39:43.201670+0000","objects_scrubbed":0,"log_size":2548,"log_dups_size":0,"ondisk_log_size":2548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:31:19.053061+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'2567","reported_seq":2611,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.264382+0000","last_change":"2024-05-09T07:39:52.264408+0000","last_active":"2024-05-09T07:39:52.264382+0000","last_peered":"2024-05-09T07:39:52.264382+0000","last_clean":"2024-05-09T07:39:52.264382+0000","last_became_active":"2024-05-09T07:13:45.486038+0000","last_became_peered":"2024-05-09T07:13:45.486038+0000","last_unstale":"2024-05-09T07:39:52.264382+0000","last_undegraded":"2024-05-09T07:39:52.264382+0000","last_fullsized":"2024-05-09T07:39:52.264382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2567","last_scrub_stamp":"2024-05-09T07:39:52.264297+0000","last_deep_scrub":"27'2567","last_deep_scrub_stamp":"2024-05-09T07:39:52.264297+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.264297+0000","objects_scrubbed":0,"log_size":2567,"log_dups_size":0,"ondisk_log_size":2567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T17:14:10.371073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2520","reported_seq":2565,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.965219+0000","last_change":"2024-05-09T07:39:52.965272+0000","last_active":"2024-05-09T07:39:52.965219+0000","last_peered":"2024-05-09T07:39:52.965219+0000","last_clean":"2024-05-09T07:39:52.965219+0000","last_became_active":"2024-05-09T07:13:45.485204+0000","last_became_peered":"2024-05-09T07:13:45.485204+0000","last_unstale":"2024-05-09T07:39:52.965219+0000","last_undegraded":"2024-05-09T07:39:52.965219+0000","last_fullsized":"2024-05-09T07:39:52.965219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2520","last_scrub_stamp":"2024-05-09T07:39:52.965125+0000","last_deep_scrub":"27'2520","last_deep_scrub_stamp":"2024-05-09T07:39:52.965125+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.965125+0000","objects_scrubbed":0,"log_size":2520,"log_dups_size":0,"ondisk_log_size":2520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:23:09.305340+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'2550","reported_seq":2594,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:55.264589+0000","last_change":"2024-05-09T07:39:55.264614+0000","last_active":"2024-05-09T07:39:55.264589+0000","last_peered":"2024-05-09T07:39:55.264589+0000","last_clean":"2024-05-09T07:39:55.264589+0000","last_became_active":"2024-05-09T07:13:45.492380+0000","last_became_peered":"2024-05-09T07:13:45.492380+0000","last_unstale":"2024-05-09T07:39:55.264589+0000","last_undegraded":"2024-05-09T07:39:55.264589+0000","last_fullsized":"2024-05-09T07:39:55.264589+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2550","last_scrub_stamp":"2024-05-09T07:39:55.264504+0000","last_deep_scrub":"27'2550","last_deep_scrub_stamp":"2024-05-09T07:39:55.264504+0000","last_clean_scrub_stamp":"2024-05-09T07:39:55.264504+0000","objects_scrubbed":0,"log_size":2550,"log_dups_size":0,"ondisk_log_size":2550,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:04:20.886229+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2592","reported_seq":2640,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:52.012614+0000","last_change":"2024-05-09T07:39:52.012614+0000","last_active":"2024-05-09T07:39:52.012614+0000","last_peered":"2024-05-09T07:39:52.012614+0000","last_clean":"2024-05-09T07:39:52.012614+0000","last_became_active":"2024-05-09T07:13:45.484510+0000","last_became_peered":"2024-05-09T07:13:45.484510+0000","last_unstale":"2024-05-09T07:39:52.012614+0000","last_undegraded":"2024-05-09T07:39:52.012614+0000","last_fullsized":"2024-05-09T07:39:52.012614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2592","last_scrub_stamp":"2024-05-09T07:39:52.012510+0000","last_deep_scrub":"27'2592","last_deep_scrub_stamp":"2024-05-09T07:39:52.012510+0000","last_clean_scrub_stamp":"2024-05-09T07:39:52.012510+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:01:37.844862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'2671","reported_seq":2717,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:44.675862+0000","last_change":"2024-05-09T07:39:44.675901+0000","last_active":"2024-05-09T07:39:44.675862+0000","last_peered":"2024-05-09T07:39:44.675862+0000","last_clean":"2024-05-09T07:39:44.675862+0000","last_became_active":"2024-05-09T07:13:45.469053+0000","last_became_peered":"2024-05-09T07:13:45.469053+0000","last_unstale":"2024-05-09T07:39:44.675862+0000","last_undegraded":"2024-05-09T07:39:44.675862+0000","last_fullsized":"2024-05-09T07:39:44.675862+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2671","last_scrub_stamp":"2024-05-09T07:39:44.675764+0000","last_deep_scrub":"27'2671","last_deep_scrub_stamp":"2024-05-09T07:39:44.675764+0000","last_clean_scrub_stamp":"2024-05-09T07:39:44.675764+0000","objects_scrubbed":0,"log_size":2671,"log_dups_size":0,"ondisk_log_size":2671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:22:44.946523+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'2502","reported_seq":2549,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:49.375269+0000","last_change":"2024-05-09T07:39:49.375292+0000","last_active":"2024-05-09T07:39:49.375269+0000","last_peered":"2024-05-09T07:39:49.375269+0000","last_clean":"2024-05-09T07:39:49.375269+0000","last_became_active":"2024-05-09T07:13:45.475347+0000","last_became_peered":"2024-05-09T07:13:45.475347+0000","last_unstale":"2024-05-09T07:39:49.375269+0000","last_undegraded":"2024-05-09T07:39:49.375269+0000","last_fullsized":"2024-05-09T07:39:49.375269+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2502","last_scrub_stamp":"2024-05-09T07:39:49.375180+0000","last_deep_scrub":"27'2502","last_deep_scrub_stamp":"2024-05-09T07:39:49.375180+0000","last_clean_scrub_stamp":"2024-05-09T07:39:49.375180+0000","objects_scrubbed":0,"log_size":2502,"log_dups_size":0,"ondisk_log_size":2502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:39:40.237981+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'2487","reported_seq":2533,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-05-09T07:39:57.290039+0000","last_change":"2024-05-09T07:39:57.290079+0000","last_active":"2024-05-09T07:39:57.290039+0000","last_peered":"2024-05-09T07:39:57.290039+0000","last_clean":"2024-05-09T07:39:57.290039+0000","last_became_active":"2024-05-09T07:13:45.492995+0000","last_became_peered":"2024-05-09T07:13:45.492995+0000","last_unstale":"2024-05-09T07:39:57.290039+0000","last_undegraded":"2024-05-09T07:39:57.290039+0000","last_fullsized":"2024-05-09T07:39:57.290039+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2487","last_scrub_stamp":"2024-05-09T07:39:57.289935+0000","last_deep_scrub":"27'2487","last_deep_scrub_stamp":"2024-05-09T07:39:57.289935+0000","last_clean_scrub_stamp":"2024-05-09T07:39:57.289935+0000","objects_scrubbed":0,"log_size":2487,"log_dups_size":0,"ondisk_log_size":2487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:11:20.500903+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12270,"num_read_kb":3117350,"num_write":214738,"num_write_kb":30722215,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232137,"ondisk_log_size":232137,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1120,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":164829,"ondisk_log_size":164829,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":248714909,"num_objects":83,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":0,"num_read":3182,"num_read_kb":19440,"num_write":211797,"num_write_kb":1695590,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":9301,"num_omap_keys":18,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":497491968,"data_stored":497429818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":193015,"ondisk_log_size":193015,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575172,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":960860,"kb_used_data":55124,"kb_used_omap":1,"kb_used_meta":905726,"kb_avail":93410980,"statfs":{"total":96636764160,"available":95652843520,"internally_reserved":0,"allocated":56446976,"data_stored":56268948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":927463882},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575171,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104412,"kb_used_data":66836,"kb_used_omap":1,"kb_used_meta":1037566,"kb_avail":93267428,"statfs":{"total":96636764160,"available":95505846272,"internally_reserved":0,"allocated":68440064,"data_stored":68261582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1062468038},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575171,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":983016,"kb_used_data":45344,"kb_used_omap":1,"kb_used_meta":937662,"kb_avail":93388824,"statfs":{"total":96636764160,"available":95630155776,"internally_reserved":0,"allocated":46432256,"data_stored":46258861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":960166341},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575170,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1043696,"kb_used_data":65832,"kb_used_omap":1,"kb_used_meta":977854,"kb_avail":93328144,"statfs":{"total":96636764160,"available":95568019456,"internally_reserved":0,"allocated":67411968,"data_stored":67230417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":1001322950},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575172,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1220228,"kb_used_data":70140,"kb_used_omap":1,"kb_used_meta":1150078,"kb_avail":93151612,"statfs":{"total":96636764160,"available":95387250688,"internally_reserved":0,"allocated":71823360,"data_stored":71644151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":1177680327},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575173,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":710908,"kb_used_data":53748,"kb_used_omap":1,"kb_used_meta":657150,"kb_avail":93660932,"statfs":{"total":96636764160,"available":95908794368,"internally_reserved":0,"allocated":55037952,"data_stored":54866263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":672922053},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575174,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1233900,"kb_used_data":65828,"kb_used_omap":1,"kb_used_meta":1168062,"kb_avail":93137940,"statfs":{"total":96636764160,"available":95373250560,"internally_reserved":0,"allocated":67407872,"data_stored":67230451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1196095941},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575172,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":899828,"kb_used_data":66412,"kb_used_omap":1,"kb_used_meta":833406,"kb_avail":93472012,"statfs":{"total":96636764160,"available":95715340288,"internally_reserved":0,"allocated":68005888,"data_stored":67821297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":853408197},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67109502,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":67117056,"data_stored":67109024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":54747136,"data_stored":54744836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":71532544,"data_stored":71522724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":67121152,"data_stored":67108990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":46141440,"data_stored":46137434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":68149248,"data_stored":68140155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":55562240,"data_stored":55557153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T07:40:25.168 DEBUG:teuthology.orchestra.run.smithi160:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-09T07:40:25.304 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.293+0000 7fafd42ee640 1 Processor -- start 2024-05-09T07:40:25.304 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.293+0000 7fafd42ee640 1 -- start start 2024-05-09T07:40:25.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd42ee640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fafcc152a80 0x7fafcc1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd42ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc159310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd42ee640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fafcc0c5690 0x7fafcc0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd42ee640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fafcc152e60 con 0x7fafcc152a80 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd42ee640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fafcc0c80a0 con 0x7fafcc0c5690 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd42ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafcc0c7ae0 con 0x7fafcc155e70 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fafcc152a80 0x7fafcc1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fafcc152a80 0x7fafcc1568e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33916/0 (socket says 172.21.15.160:33916) 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 -- 172.21.15.160:0/2180696002 learned_addr learned my addr 172.21.15.160:0/2180696002 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd1889640 1 --2- 172.21.15.160:0/2180696002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc159310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd288b640 1 --2- 172.21.15.160:0/2180696002 >> v2:172.21.15.188:3300/0 conn(0x7fafcc0c5690 0x7fafcc0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.306 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 -- 172.21.15.160:0/2180696002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 msgr2=0x7fafcc159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 --2- 172.21.15.160:0/2180696002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc159310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 -- 172.21.15.160:0/2180696002 >> v2:172.21.15.188:3300/0 conn(0x7fafcc0c5690 msgr2=0x7fafcc0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 --2- 172.21.15.160:0/2180696002 >> v2:172.21.15.188:3300/0 conn(0x7fafcc0c5690 0x7fafcc0c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.294+0000 7fafd208a640 1 -- 172.21.15.160:0/2180696002 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc0c7dc0 con 0x7fafcc152a80 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.295+0000 7fafd208a640 1 --2- 172.21.15.160:0/2180696002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc1568e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fafcc06d8f0 tx=0x7fafc802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9707769a7a08e0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.295+0000 7fafd1088640 1 -- 172.21.15.160:0/2180696002 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8037070 con 0x7fafcc152a80 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.295+0000 7fafd1088640 1 -- 172.21.15.160:0/2180696002 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafc802ae00 con 0x7fafcc152a80 2024-05-09T07:40:25.307 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.295+0000 7fafd1088640 1 -- 172.21.15.160:0/2180696002 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8032660 con 0x7fafcc152a80 2024-05-09T07:40:25.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.295+0000 7fafd42ee640 1 -- 172.21.15.160:0/2180696002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 msgr2=0x7fafcc1568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.295+0000 7fafd42ee640 1 --2- 172.21.15.160:0/2180696002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc1568e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fafcc06d8f0 tx=0x7fafc802a0b0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 -- 172.21.15.160:0/2180696002 shutdown_connections 2024-05-09T07:40:25.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 --2- 172.21.15.160:0/2180696002 >> v2:172.21.15.188:3300/0 conn(0x7fafcc0c5690 0x7fafcc0c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.310 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 --2- 172.21.15.160:0/2180696002 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc159310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 --2- 172.21.15.160:0/2180696002 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc1568e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 -- 172.21.15.160:0/2180696002 >> 172.21.15.160:0/2180696002 conn(0x7fafcc072ec0 msgr2=0x7fafcc0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 -- 172.21.15.160:0/2180696002 shutdown_connections 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.296+0000 7fafd42ee640 1 -- 172.21.15.160:0/2180696002 wait complete. 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 Processor -- start 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 -- start start 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafcc0c5690 0x7fafcc1ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc0c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc0c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc156320 con 0x7fafcc152a80 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fafcc158df0 con 0x7fafcc0c5690 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd42ee640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fafcc0ad6a0 con 0x7fafcc155e70 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.297+0000 7fafd1889640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc0c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc0c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33924/0 (socket says 172.21.15.160:33924) 2024-05-09T07:40:25.311 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 -- 172.21.15.160:0/3628002213 learned_addr learned my addr 172.21.15.160:0/3628002213 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd208a640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafcc0c5690 0x7fafcc1ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd288b640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc0c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 -- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 msgr2=0x7fafcc0c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc0c9fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 -- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafcc0c5690 msgr2=0x7fafcc1ee9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafcc0c5690 0x7fafcc1ee9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 -- 172.21.15.160:0/3628002213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafcc0ca920 con 0x7fafcc152a80 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd288b640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc0c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd1889640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc0c9a80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00d800 tx=0x7fafbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00b6c0 con 0x7fafcc152a80 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fafbc019070 con 0x7fafcc152a80 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafbc00ba80 con 0x7fafcc152a80 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafcc0c95a0 con 0x7fafcc152a80 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.298+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafcc1ef2f0 con 0x7fafcc152a80 2024-05-09T07:40:25.312 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.299+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fafcc0ad3a0 con 0x7fafcc152a80 2024-05-09T07:40:25.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.300+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fafbc00bc20 con 0x7fafcc152a80 2024-05-09T07:40:25.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.301+0000 7fafc2ffd640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fafb006be30 0x7fafb006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.301+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fafbc08b690 con 0x7fafcc152a80 2024-05-09T07:40:25.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.301+0000 7fafd208a640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fafb006be30 0x7fafb006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.313 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.301+0000 7fafd208a640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fafb006be30 0x7fafb006e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fafc8034000 tx=0x7fafc802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.316 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.305+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fafcc0ca920 con 0x7fafcc152a80 2024-05-09T07:40:25.504 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.492+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fafcc075630 con 0x7fafcc152a80 2024-05-09T07:40:25.506 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.495+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) ==== 123+0+0 (secure 0 0 0) 0x7fafbc059ce0 con 0x7fafcc152a80 2024-05-09T07:40:25.508 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.497+0000 7fafc2ffd640 1 -- 172.21.15.160:0/3628002213 <== mon.0 v2:172.21.15.160:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fafbc0595e0 con 0x7fafcc152a80 2024-05-09T07:40:25.509 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fafb006be30 msgr2=0x7fafb006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fafb006be30 0x7fafb006e2f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fafc8034000 tx=0x7fafc802e000 comp rx=0 tx=0).stop 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 msgr2=0x7fafcc0c9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc0c9a80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fafbc00d800 tx=0x7fafbc00dcc0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 shutdown_connections 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fafb006be30 0x7fafb006e2f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fafcc155e70 0x7fafcc0c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fafcc152a80 0x7fafcc0c9a80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 --2- 172.21.15.160:0/3628002213 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fafcc0c5690 0x7fafcc1ee9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.498+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 >> 172.21.15.160:0/3628002213 conn(0x7fafcc072ec0 msgr2=0x7fafcc150050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.499+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 shutdown_connections 2024-05-09T07:40:25.510 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.499+0000 7fafd42ee640 1 -- 172.21.15.160:0/3628002213 wait complete. 2024-05-09T07:40:25.521 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T07:40:25.521 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:40:25.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.612+0000 7f65633e9640 1 Processor -- start 2024-05-09T07:40:25.623 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.612+0000 7f65633e9640 1 -- start start 2024-05-09T07:40:25.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f65633e9640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f655c0c6640 0x7f655c072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f65633e9640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c152320 con 0x7f655c156040 2024-05-09T07:40:25.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f65633e9640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f655c152600 con 0x7f655c0c2450 2024-05-09T07:40:25.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f65633e9640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f655c14fbf0 con 0x7f655c0c6640 2024-05-09T07:40:25.624 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f6560984640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f655c0c6640 0x7f655c072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f6561986640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7f655c156040 0x7f655c15c9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52058/0 (socket says 172.21.15.160:52058) 2024-05-09T07:40:25.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f6561986640 1 -- 172.21.15.160:0/304282724 learned_addr learned my addr 172.21.15.160:0/304282724 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:25.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f6560984640 1 -- 172.21.15.160:0/304282724 >> v1:172.21.15.188:6789/0 conn(0x7f655c0c2450 legacy=0x7f655c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f6560984640 1 -- 172.21.15.160:0/304282724 >> v1:172.21.15.160:6789/0 conn(0x7f655c156040 legacy=0x7f655c15c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.614+0000 7f6560984640 1 -- 172.21.15.160:0/304282724 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c1528e0 con 0x7f655c0c6640 2024-05-09T07:40:25.625 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.613+0000 7f6553fff640 1 -- 172.21.15.160:0/304282724 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3099025201 0 0) 0x7f655c152320 con 0x7f655c156040 2024-05-09T07:40:25.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.614+0000 7f6560984640 1 --2- 172.21.15.160:0/304282724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c0c6640 0x7f655c072080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f65580097a0 tx=0x7f655802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f8104b760f24b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.614+0000 7f6553fff640 1 -- 172.21.15.160:0/304282724 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558037070 con 0x7f655c0c6640 2024-05-09T07:40:25.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.614+0000 7f6553fff640 1 -- 172.21.15.160:0/304282724 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f6558032440 con 0x7f655c0c6640 2024-05-09T07:40:25.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.614+0000 7f6553fff640 1 -- 172.21.15.160:0/304282724 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6558032830 con 0x7f655c0c6640 2024-05-09T07:40:25.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 -- 172.21.15.160:0/304282724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c0c6640 msgr2=0x7f655c072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.626 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 --2- 172.21.15.160:0/304282724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c0c6640 0x7f655c072080 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f65580097a0 tx=0x7f655802a230 comp rx=0 tx=0).stop 2024-05-09T07:40:25.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 -- 172.21.15.160:0/304282724 shutdown_connections 2024-05-09T07:40:25.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 --2- 172.21.15.160:0/304282724 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c0c6640 0x7f655c072080 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 -- 172.21.15.160:0/304282724 >> 172.21.15.160:0/304282724 conn(0x7f655c0a9800 msgr2=0x7f655c0a9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:25.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 -- 172.21.15.160:0/304282724 shutdown_connections 2024-05-09T07:40:25.627 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.615+0000 7f65633e9640 1 -- 172.21.15.160:0/304282724 wait complete. 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.616+0000 7f65633e9640 1 Processor -- start 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.616+0000 7f65633e9640 1 -- start start 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f65633e9640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f655c0c2450 0x7f655c1e9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f65633e9640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f655c0c6640 0x7f655c1dc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f65633e9640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 0x7f655c153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f65633e9640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f655c152c50 con 0x7f655c156040 2024-05-09T07:40:25.629 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f65633e9640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f655c072ff0 con 0x7f655c0c2450 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f65633e9640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f655c073170 con 0x7f655c0c6640 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 0x7f655c153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 0x7f655c153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33946/0 (socket says 172.21.15.160:33946) 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 -- 172.21.15.160:0/2566654700 learned_addr learned my addr 172.21.15.160:0/2566654700 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561185640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f655c0c2450 0x7f655c1e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6560984640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f655c0c6640 0x7f655c1dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 -- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f655c0c6640 msgr2=0x7f655c1dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f655c0c6640 0x7f655c1dc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 -- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f655c0c2450 msgr2=0x7f655c1e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f655c0c2450 0x7f655c1e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 -- 172.21.15.160:0/2566654700 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c0bd060 con 0x7f655c156040 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.617+0000 7f6561986640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 0x7f655c153ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f655400c7a0 tx=0x7f655400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.618+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554018070 con 0x7f655c156040 2024-05-09T07:40:25.630 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.618+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f655400a640 con 0x7f655c156040 2024-05-09T07:40:25.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.618+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c1e9950 con 0x7f655c156040 2024-05-09T07:40:25.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.618+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655400aad0 con 0x7f655c156040 2024-05-09T07:40:25.631 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.618+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c1e9f10 con 0x7f655c156040 2024-05-09T07:40:25.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.619+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655c0756c0 con 0x7f655c156040 2024-05-09T07:40:25.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.619+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f655401b070 con 0x7f655c156040 2024-05-09T07:40:25.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.620+0000 7f6551ffb640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f654006be30 0x7f654006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.620+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f655408a710 con 0x7f655c156040 2024-05-09T07:40:25.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.620+0000 7f6561185640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f654006be30 0x7f654006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.632 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.620+0000 7f6561185640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f654006be30 0x7f654006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f655c1505f0 tx=0x7f654c009040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.635 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.624+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6554058710 con 0x7f655c156040 2024-05-09T07:40:25.854 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.843+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655c0a95e0 con 0x7f655c156040 2024-05-09T07:40:25.855 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.844+0000 7f6551ffb640 1 -- 172.21.15.160:0/2566654700 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7f655405ca10 con 0x7f655c156040 2024-05-09T07:40:25.855 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:40:25.855 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:14:00.864206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:40:25.855 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-05-09T07:40:25.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f654006be30 msgr2=0x7f654006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.857 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f654006be30 0x7f654006e2f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f655c1505f0 tx=0x7f654c009040 comp rx=0 tx=0).stop 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 msgr2=0x7f655c153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 0x7f655c153ab0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f655400c7a0 tx=0x7f655400cc60 comp rx=0 tx=0).stop 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 shutdown_connections 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f654006be30 0x7f654006e2f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f655c156040 0x7f655c153ab0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f655c0c6640 0x7f655c1dc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 --2- 172.21.15.160:0/2566654700 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f655c0c2450 0x7f655c1e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 >> 172.21.15.160:0/2566654700 conn(0x7f655c0a9800 msgr2=0x7f655c150940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 shutdown_connections 2024-05-09T07:40:25.858 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.846+0000 7f65633e9640 1 -- 172.21.15.160:0/2566654700 wait complete. 2024-05-09T07:40:25.869 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.960+0000 7fe455ad6640 1 Processor -- start 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe455ad6640 1 -- start start 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe455ad6640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4500c5610 0x7fe4500c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe455ad6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe450155a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe455ad6640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe450159840 con 0x7fe4500be320 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe455ad6640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe450159310 con 0x7fe4500c5610 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe455ad6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe450159560 con 0x7fe4500c8b90 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.961+0000 7fe447fff640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe4500be320 0x7fe450155310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52060/0 (socket says 172.21.15.160:52060) 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe447fff640 1 -- 172.21.15.160:0/1117874604 learned_addr learned my addr 172.21.15.160:0/1117874604 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:25.981 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44ffff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe450155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 193412496 0 0) 0x7fe450159840 con 0x7fe4500be320 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe43c003520 con 0x7fe4500be320 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44f7fe640 1 --2- 172.21.15.160:0/1117874604 >> v2:172.21.15.188:3300/0 conn(0x7fe4500c5610 0x7fe4500c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3500815936 0 0) 0x7fe43c003520 con 0x7fe4500be320 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 msgr2=0x7fe450155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe450155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 >> v2:172.21.15.188:3300/0 conn(0x7fe4500c5610 msgr2=0x7fe4500c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 --2- 172.21.15.160:0/1117874604 >> v2:172.21.15.188:3300/0 conn(0x7fe4500c5610 0x7fe4500c59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500c77d0 con 0x7fe4500be320 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7fe438001670 con 0x7fe4500be320 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 >> v1:172.21.15.160:6789/0 conn(0x7fe4500be320 legacy=0x7fe450155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe43c0039b0 0x7fe43c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.982 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe43c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe450159310 con 0x7fe43c004800 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe450159560 con 0x7fe43c0039b0 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe43c00ba40 con 0x7fe4500c8b90 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe447fff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44ffff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe43c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.962+0000 7fe44f7fe640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe43c0039b0 0x7fe43c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe447fff640 1 -- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 msgr2=0x7fe43c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe447fff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe43c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe447fff640 1 -- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe43c0039b0 msgr2=0x7fe43c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe447fff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe43c0039b0 0x7fe43c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe447fff640 1 -- 172.21.15.160:0/1117874604 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe43c003520 con 0x7fe43c004800 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe44ffff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4500c8b90 0x7fe43c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe447fff640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4500be760 tx=0x7fe43c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b854414c65c7cfe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438007d50 con 0x7fe43c004800 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe4380058d0 con 0x7fe43c004800 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.963+0000 7fe44effd640 1 -- 172.21.15.160:0/1117874604 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe438005ce0 con 0x7fe43c004800 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe455ad6640 1 -- 172.21.15.160:0/1117874604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 msgr2=0x7fe43c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe455ad6640 1 --2- 172.21.15.160:0/1117874604 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe43c004c80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe4500be760 tx=0x7fe43c005830 comp rx=0 tx=0).stop 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe44ffff640 1 -- 172.21.15.160:0/1117874604 reap_dead start 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe455ad6640 1 -- 172.21.15.160:0/1117874604 shutdown_connections 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe455ad6640 1 -- 172.21.15.160:0/1117874604 >> 172.21.15.160:0/1117874604 conn(0x7fe450072e90 msgr2=0x7fe4500ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe455ad6640 1 -- 172.21.15.160:0/1117874604 shutdown_connections 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.964+0000 7fe455ad6640 1 -- 172.21.15.160:0/1117874604 wait complete. 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.965+0000 7fe455ad6640 1 Processor -- start 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.965+0000 7fe455ad6640 1 -- start start 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe455ad6640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe4501ee9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe455ad6640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4501eef10 0x7fe4500c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe455ad6640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4501ec660 0x7fe45014ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe455ad6640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe4500ad6a0 con 0x7fe43c004800 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe455ad6640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe4500c7d90 con 0x7fe4501ec660 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe455ad6640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe450152bb0 con 0x7fe4501eef10 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe4501ee9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe4501ee9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33962/0 (socket says 172.21.15.160:33962) 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 -- 172.21.15.160:0/3855663547 learned_addr learned my addr 172.21.15.160:0/3855663547 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe447fff640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4501eef10 0x7fe4500c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 -- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4501eef10 msgr2=0x7fe4500c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.985 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44ffff640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4501ec660 0x7fe45014ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4501eef10 0x7fe4500c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 -- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4501ec660 msgr2=0x7fe45014ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4501ec660 0x7fe45014ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.966+0000 7fe44f7fe640 1 -- 172.21.15.160:0/3855663547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4500c3b30 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.967+0000 7fe44f7fe640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe4501ee9d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe440011540 tx=0x7fe440011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.967+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe440015930 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.967+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe440015ad0 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.967+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4500c4a30 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.967+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe44000f460 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.967+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4500c28a0 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.968+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4500756c0 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.969+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe44000f600 con 0x7fe43c004800 2024-05-09T07:40:25.986 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.969+0000 7fe44cff9640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe43006be30 0x7fe43006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:25.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.969+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe44008baa0 con 0x7fe43c004800 2024-05-09T07:40:25.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.969+0000 7fe447fff640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe43006be30 0x7fe43006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:25.987 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.970+0000 7fe447fff640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe43006be30 0x7fe43006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe438033000 tx=0x7fe438009030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:25.988 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:25.977+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe440019030 con 0x7fe43c004800 2024-05-09T07:40:26.206 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.195+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4500ad340 con 0x7fe43c004800 2024-05-09T07:40:26.207 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.196+0000 7fe44cff9640 1 -- 172.21.15.160:0/3855663547 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5183 (secure 0 0 0) 0x7fe44005a0f0 con 0x7fe43c004800 2024-05-09T07:40:26.207 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:40:26.207 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.188:6835/4293795835","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6834","nonce":4293795835},{"type":"v1","addr":"172.21.15.188:6835","nonce":4293795835}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4284,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/145206445","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6835","nonce":145206445},{"type":"v1","addr":"172.21.15.160:6837","nonce":145206445}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6836/216550135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":216550135},{"type":"v1","addr":"172.21.15.160:6836","nonce":216550135}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-09T07:13:44.804208+0000","modified":"2024-05-09T07:14:00.864206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.188:6833/3545719682","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6832","nonce":3545719682},{"type":"v1","addr":"172.21.15.188:6833","nonce":3545719682}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4296,"qdb_cluster":[4296]},"id":1}]} 2024-05-09T07:40:26.207 INFO:teuthology.orchestra.run.smithi160.stderr:dumped fsmap epoch 8 2024-05-09T07:40:26.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe43006be30 msgr2=0x7fe43006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.209 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe43006be30 0x7fe43006e2f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe438033000 tx=0x7fe438009030 comp rx=0 tx=0).stop 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 msgr2=0x7fe4501ee9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe4501ee9d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe440011540 tx=0x7fe440011a00 comp rx=0 tx=0).stop 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 shutdown_connections 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe43006be30 0x7fe43006e2f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4501ec660 0x7fe45014ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4501eef10 0x7fe4500c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 --2- 172.21.15.160:0/3855663547 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe43c004800 0x7fe4501ee9d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 >> 172.21.15.160:0/3855663547 conn(0x7fe450072e90 msgr2=0x7fe4500aa9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 shutdown_connections 2024-05-09T07:40:26.210 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.198+0000 7fe455ad6640 1 -- 172.21.15.160:0/3855663547 wait complete. 2024-05-09T07:40:26.221 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.311+0000 7f41b4e64640 1 Processor -- start 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.311+0000 7f41b4e64640 1 -- start start 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41b4e64640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f41b00c5610 0x7f41b00c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41b4e64640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f41b00be320 0x7f41b0155310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41b4e64640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41b0159840 con 0x7f41b00c8b90 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41b4e64640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7f41b0159310 con 0x7f41b00c5610 2024-05-09T07:40:26.324 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41b4e64640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f41b0159560 con 0x7f41b00be320 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41ae59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f41b00c5610 0x7f41b00c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41ae59c640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7f41b00c5610 0x7f41b00c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33978/0 (socket says 172.21.15.160:33978) 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.312+0000 7f41ae59c640 1 -- 172.21.15.160:0/3971178874 learned_addr learned my addr 172.21.15.160:0/3971178874 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41add9b640 1 --2- 172.21.15.160:0/3971178874 >> v2:172.21.15.188:3300/0 conn(0x7f41b00be320 0x7f41b0155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41ae59c640 1 -- 172.21.15.160:0/3971178874 >> v2:172.21.15.188:3300/0 conn(0x7f41b00be320 msgr2=0x7f41b0155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41ae59c640 1 --2- 172.21.15.160:0/3971178874 >> v2:172.21.15.188:3300/0 conn(0x7f41b00be320 0x7f41b0155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41ae59c640 1 -- 172.21.15.160:0/3971178874 >> v1:172.21.15.188:6789/0 conn(0x7f41b00c8b90 legacy=0x7f41b0155850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41ad59a640 1 -- 172.21.15.160:0/3971178874 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1786778855 0 0) 0x7f41b0159840 con 0x7f41b00c8b90 2024-05-09T07:40:26.325 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41ae59c640 1 -- 172.21.15.160:0/3971178874 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b00c77d0 con 0x7f41b00c5610 2024-05-09T07:40:26.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.313+0000 7f41ae59c640 1 --2- 172.21.15.160:0/3971178874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c5610 0x7f41b00c59f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41b006d860 tx=0x7f41a002a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846f71614f3dbc6d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:26.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41ad59a640 1 -- 172.21.15.160:0/3971178874 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0037070 con 0x7f41b00c5610 2024-05-09T07:40:26.328 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41ad59a640 1 -- 172.21.15.160:0/3971178874 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f41a0032440 con 0x7f41b00c5610 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41ad59a640 1 -- 172.21.15.160:0/3971178874 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0032830 con 0x7f41b00c5610 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 -- 172.21.15.160:0/3971178874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c5610 msgr2=0x7f41b00c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 --2- 172.21.15.160:0/3971178874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c5610 0x7f41b00c59f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f41b006d860 tx=0x7f41a002a150 comp rx=0 tx=0).stop 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 -- 172.21.15.160:0/3971178874 shutdown_connections 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 --2- 172.21.15.160:0/3971178874 >> v2:172.21.15.188:3300/0 conn(0x7f41b00be320 0x7f41b0155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 --2- 172.21.15.160:0/3971178874 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c5610 0x7f41b00c59f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 -- 172.21.15.160:0/3971178874 >> 172.21.15.160:0/3971178874 conn(0x7f41b0072e90 msgr2=0x7f41b00ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 -- 172.21.15.160:0/3971178874 shutdown_connections 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.314+0000 7f41b4e64640 1 -- 172.21.15.160:0/3971178874 wait complete. 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.315+0000 7f41b4e64640 1 Processor -- start 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 -- start start 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b00be320 0x7f41b01eb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.329 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b00c5610 0x7f41b00ab950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 0x7f41b00c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f41b00ad6a0 con 0x7f41b00c8b90 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f41b00c7d90 con 0x7f41b00c5610 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41b4e64640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f41b0155f60 con 0x7f41b00be320 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41aed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 0x7f41b00c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41aed9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 0x7f41b00c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33980/0 (socket says 172.21.15.160:33980) 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.316+0000 7f41aed9d640 1 -- 172.21.15.160:0/2826224412 learned_addr learned my addr 172.21.15.160:0/2826224412 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41ae59c640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b00be320 0x7f41b01eb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41add9b640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b00c5610 0x7f41b00ab950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41aed9d640 1 -- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b00be320 msgr2=0x7f41b01eb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41aed9d640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b00be320 0x7f41b01eb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41aed9d640 1 -- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b00c5610 msgr2=0x7f41b00ab950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41aed9d640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b00c5610 0x7f41b00ab950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.330 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41aed9d640 1 -- 172.21.15.160:0/2826224412 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41b00c90f0 con 0x7f41b00c8b90 2024-05-09T07:40:26.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41aed9d640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 0x7f41b00c98b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c7b0 tx=0x7f41a800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:26.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8003ba0 con 0x7f41b00c8b90 2024-05-09T07:40:26.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f41a8003d40 con 0x7f41b00c8b90 2024-05-09T07:40:26.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41b00ca770 con 0x7f41b00c8b90 2024-05-09T07:40:26.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a8005290 con 0x7f41b00c8b90 2024-05-09T07:40:26.331 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.317+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41b00cad30 con 0x7f41b00c8b90 2024-05-09T07:40:26.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.318+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41b00756c0 con 0x7f41b00c8b90 2024-05-09T07:40:26.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.319+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f41a8005430 con 0x7f41b00c8b90 2024-05-09T07:40:26.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.319+0000 7f41a6ffd640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f419006be30 0x7f419006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.319+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7f41a808bce0 con 0x7f41b00c8b90 2024-05-09T07:40:26.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.319+0000 7f41ae59c640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f419006be30 0x7f419006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.332 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.320+0000 7f41ae59c640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f419006be30 0x7f419006e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41a0034000 tx=0x7f41a002e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:26.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.323+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f41a805a210 con 0x7f41b00c8b90 2024-05-09T07:40:26.519 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.508+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f41b00ad340 con 0x7f41b00c8b90 2024-05-09T07:40:26.520 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.510+0000 7f41a6ffd640 1 -- 172.21.15.160:0/2826224412 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18909 (secure 0 0 0) 0x7f41a8059b10 con 0x7f41b00c8b90 2024-05-09T07:40:26.521 INFO:teuthology.orchestra.run.smithi160.stdout: 2024-05-09T07:40:26.521 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":27,"fsid":"22e9edb5-902d-460d-aabf-1eff035c9624","created":"2024-05-09T07:13:21.015759+0000","modified":"2024-05-09T07:14:26.582005+0000","last_up_change":"2024-05-09T07:13:32.292693+0000","last_in_change":"2024-05-09T07:13:23.266097+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T07:13:33.210504+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T07:13:43.238490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T07:13:43.781116+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-09T07:13:47.498198+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"64c086fb-92b0-426f-88dc-821957a5704b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6801","nonce":3071505742}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6803","nonce":3071505742}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6807","nonce":3071505742}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3071505742},{"type":"v1","addr":"172.21.15.160:6805","nonce":3071505742}]},"public_addr":"172.21.15.160:6801/3071505742","cluster_addr":"172.21.15.160:6803/3071505742","heartbeat_back_addr":"172.21.15.160:6807/3071505742","heartbeat_front_addr":"172.21.15.160:6805/3071505742","state":["exists","up"]},{"osd":1,"uuid":"348cc74d-e143-48f9-a79f-225bd020ac74","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6817","nonce":4171986763}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6819","nonce":4171986763}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6823","nonce":4171986763}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":4171986763},{"type":"v1","addr":"172.21.15.160:6821","nonce":4171986763}]},"public_addr":"172.21.15.160:6817/4171986763","cluster_addr":"172.21.15.160:6819/4171986763","heartbeat_back_addr":"172.21.15.160:6823/4171986763","heartbeat_front_addr":"172.21.15.160:6821/4171986763","state":["exists","up"]},{"osd":2,"uuid":"a146b295-ee51-4e60-9ec9-aa7880de8049","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6809","nonce":2850827028}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6811","nonce":2850827028}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6815","nonce":2850827028}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":2850827028},{"type":"v1","addr":"172.21.15.160:6813","nonce":2850827028}]},"public_addr":"172.21.15.160:6809/2850827028","cluster_addr":"172.21.15.160:6811/2850827028","heartbeat_back_addr":"172.21.15.160:6815/2850827028","heartbeat_front_addr":"172.21.15.160:6813/2850827028","state":["exists","up"]},{"osd":3,"uuid":"1ff30033-a2e4-4b39-962b-c6dab56a96b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6825","nonce":3093825945}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6827","nonce":3093825945}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6831","nonce":3093825945}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":3093825945},{"type":"v1","addr":"172.21.15.160:6829","nonce":3093825945}]},"public_addr":"172.21.15.160:6825/3093825945","cluster_addr":"172.21.15.160:6827/3093825945","heartbeat_back_addr":"172.21.15.160:6831/3093825945","heartbeat_front_addr":"172.21.15.160:6829/3093825945","state":["exists","up"]},{"osd":4,"uuid":"11b9a811-6789-4a48-aab8-e9066ea7440f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6800","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6801","nonce":2865253491}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6802","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6803","nonce":2865253491}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6806","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6807","nonce":2865253491}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6804","nonce":2865253491},{"type":"v1","addr":"172.21.15.188:6805","nonce":2865253491}]},"public_addr":"172.21.15.188:6801/2865253491","cluster_addr":"172.21.15.188:6803/2865253491","heartbeat_back_addr":"172.21.15.188:6807/2865253491","heartbeat_front_addr":"172.21.15.188:6805/2865253491","state":["exists","up"]},{"osd":5,"uuid":"a4fc7ad0-cf97-4459-84e5-f63ad1e45658","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6816","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6817","nonce":173821344}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6818","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6819","nonce":173821344}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6822","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6823","nonce":173821344}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6820","nonce":173821344},{"type":"v1","addr":"172.21.15.188:6821","nonce":173821344}]},"public_addr":"172.21.15.188:6817/173821344","cluster_addr":"172.21.15.188:6819/173821344","heartbeat_back_addr":"172.21.15.188:6823/173821344","heartbeat_front_addr":"172.21.15.188:6821/173821344","state":["exists","up"]},{"osd":6,"uuid":"894aeb3c-d190-4d55-85f0-908ba67e54ca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6824","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6825","nonce":566542668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6826","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6827","nonce":566542668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6830","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6831","nonce":566542668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6828","nonce":566542668},{"type":"v1","addr":"172.21.15.188:6829","nonce":566542668}]},"public_addr":"172.21.15.188:6825/566542668","cluster_addr":"172.21.15.188:6827/566542668","heartbeat_back_addr":"172.21.15.188:6831/566542668","heartbeat_front_addr":"172.21.15.188:6829/566542668","state":["exists","up"]},{"osd":7,"uuid":"4671e883-f5b6-46de-9280-7b1297eca4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6808","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6809","nonce":4113254918}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6810","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6811","nonce":4113254918}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6814","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6815","nonce":4113254918}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.188:6812","nonce":4113254918},{"type":"v1","addr":"172.21.15.188:6813","nonce":4113254918}]},"public_addr":"172.21.15.188:6809/4113254918","cluster_addr":"172.21.15.188:6811/4113254918","heartbeat_back_addr":"172.21.15.188:6815/4113254918","heartbeat_front_addr":"172.21.15.188:6813/4113254918","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.352539+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.433981+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.455818+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.486574+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.404375+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.408486+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.387222+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T07:13:28.369829+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2},{"from":3,"to":1}]},{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"4.f","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.13","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f419006be30 msgr2=0x7f419006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f419006be30 0x7f419006e2f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f41a0034000 tx=0x7f41a002e000 comp rx=0 tx=0).stop 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 msgr2=0x7f41b00c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 0x7f41b00c98b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f41a800c7b0 tx=0x7f41a800cbb0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 shutdown_connections 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f419006be30 0x7f419006e2f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.523 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f41b00c5610 0x7f41b00ab950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f41b00c8b90 0x7f41b00c98b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 --2- 172.21.15.160:0/2826224412 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f41b00be320 0x7f41b01eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 >> 172.21.15.160:0/2826224412 conn(0x7f41b0072e90 msgr2=0x7f41b0151690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:26.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 shutdown_connections 2024-05-09T07:40:26.524 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.512+0000 7f41b4e64640 1 -- 172.21.15.160:0/2826224412 wait complete. 2024-05-09T07:40:26.535 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.708+0000 7fe46d9ab640 1 Processor -- start 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.708+0000 7fe46d9ab640 1 -- start start 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe46d9ab640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe46814bd30 0x7fe4680c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe46d9ab640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fe4681560e0 0x7fe4680a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe46d9ab640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4680c5f30 con 0x7fe4680a7300 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe46d9ab640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fe46806ba90 con 0x7fe46814bd30 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe46d9ab640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fe4680c61d0 con 0x7fe4681560e0 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe466ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe46814bd30 0x7fe4680c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe466ffd640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fe46814bd30 0x7fe4680c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33992/0 (socket says 172.21.15.160:33992) 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe4677fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fe4680a7300 0x7fe4680c0f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52064/0 (socket says 172.21.15.160:52064) 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.709+0000 7fe466ffd640 1 -- 172.21.15.160:0/48111839 learned_addr learned my addr 172.21.15.160:0/48111839 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.710+0000 7fe466ffd640 1 -- 172.21.15.160:0/48111839 >> v2:172.21.15.188:3300/0 conn(0x7fe4681560e0 msgr2=0x7fe4680a6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:26.723 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.710+0000 7fe466ffd640 1 --2- 172.21.15.160:0/48111839 >> v2:172.21.15.188:3300/0 conn(0x7fe4681560e0 0x7fe4680a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.710+0000 7fe466ffd640 1 -- 172.21.15.160:0/48111839 >> v1:172.21.15.160:6789/0 conn(0x7fe4680a7300 legacy=0x7fe4680c0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.710+0000 7fe465ffb640 1 -- 172.21.15.160:0/48111839 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1298867737 0 0) 0x7fe4680c5f30 con 0x7fe4680a7300 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.710+0000 7fe466ffd640 1 -- 172.21.15.160:0/48111839 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe468155600 con 0x7fe46814bd30 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.710+0000 7fe466ffd640 1 --2- 172.21.15.160:0/48111839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe4680c59f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe4680aa540 tx=0x7fe45802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceae2b46de83494 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe465ffb640 1 -- 172.21.15.160:0/48111839 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458037070 con 0x7fe46814bd30 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe465ffb640 1 -- 172.21.15.160:0/48111839 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe458032440 con 0x7fe46814bd30 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe465ffb640 1 -- 172.21.15.160:0/48111839 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458032830 con 0x7fe46814bd30 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 -- 172.21.15.160:0/48111839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 msgr2=0x7fe4680c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/48111839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe4680c59f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe4680aa540 tx=0x7fe45802a270 comp rx=0 tx=0).stop 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 -- 172.21.15.160:0/48111839 shutdown_connections 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/48111839 >> v2:172.21.15.188:3300/0 conn(0x7fe4681560e0 0x7fe4680a6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/48111839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe4680c59f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.724 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 -- 172.21.15.160:0/48111839 >> 172.21.15.160:0/48111839 conn(0x7fe4680a98a0 msgr2=0x7fe4680a9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:26.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 -- 172.21.15.160:0/48111839 shutdown_connections 2024-05-09T07:40:26.725 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.711+0000 7fe46d9ab640 1 -- 172.21.15.160:0/48111839 wait complete. 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.712+0000 7fe46d9ab640 1 Processor -- start 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 -- start start 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4680a7300 0x7fe46814ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe468144190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4681560e0 0x7fe4680abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fe46814c110 con 0x7fe46814bd30 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fe468155bc0 con 0x7fe4681560e0 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe46d9ab640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fe46814c290 con 0x7fe4680a7300 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe4667fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe468144190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe4667fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe468144190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33996/0 (socket says 172.21.15.160:33996) 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe4667fc640 1 -- 172.21.15.160:0/1832630868 learned_addr learned my addr 172.21.15.160:0/1832630868 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:26.728 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe466ffd640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4680a7300 0x7fe46814ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.713+0000 7fe4677fe640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4681560e0 0x7fe4680abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe4667fc640 1 -- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4680a7300 msgr2=0x7fe46814ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe4667fc640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4680a7300 0x7fe46814ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe4667fc640 1 -- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4681560e0 msgr2=0x7fe4680abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe4667fc640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4681560e0 0x7fe4680abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe4667fc640 1 -- 172.21.15.160:0/1832630868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4680c2b80 con 0x7fe46814bd30 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe466ffd640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4680a7300 0x7fe46814ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe4667fc640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe468144190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b410 tx=0x7fe45c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c015020 con 0x7fe46814bd30 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fe45c011850 con 0x7fe46814bd30 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4680c2e20 con 0x7fe46814bd30 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe45c011c60 con 0x7fe46814bd30 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.714+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4680703b0 con 0x7fe46814bd30 2024-05-09T07:40:26.729 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.715+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4680756c0 con 0x7fe46814bd30 2024-05-09T07:40:26.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.715+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fe45c018020 con 0x7fe46814bd30 2024-05-09T07:40:26.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.716+0000 7fe44ffff640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe45406be30 0x7fe45406e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:26.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.716+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6714+0+0 (secure 0 0 0) 0x7fe45c08b1b0 con 0x7fe46814bd30 2024-05-09T07:40:26.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.716+0000 7fe466ffd640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe45406be30 0x7fe45406e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:26.730 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.716+0000 7fe466ffd640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe45406be30 0x7fe45406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe458034000 tx=0x7fe45802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:26.731 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.720+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe45c01f020 con 0x7fe46814bd30 2024-05-09T07:40:26.949 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.938+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe46806f7a0 con 0x7fe46814bd30 2024-05-09T07:40:26.951 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.940+0000 7fe44ffff640 1 -- 172.21.15.160:0/1832630868 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7fe45c0172c0 con 0x7fe46814bd30 2024-05-09T07:40:26.951 INFO:teuthology.orchestra.run.smithi160.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T07:40:26.954 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe45406be30 msgr2=0x7fe45406e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe45406be30 0x7fe45406e2f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe458034000 tx=0x7fe45802e000 comp rx=0 tx=0).stop 2024-05-09T07:40:26.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 msgr2=0x7fe468144190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:26.955 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe468144190 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00b410 tx=0x7fe45c00b8d0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 shutdown_connections 2024-05-09T07:40:26.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fe45406be30 0x7fe45406e2f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fe4681560e0 0x7fe4680abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.956 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe46814bd30 0x7fe468144190 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 --2- 172.21.15.160:0/1832630868 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fe4680a7300 0x7fe46814ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:26.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 >> 172.21.15.160:0/1832630868 conn(0x7fe4680a98a0 msgr2=0x7fe468153260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:26.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 shutdown_connections 2024-05-09T07:40:26.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:26.944+0000 7fe46d9ab640 1 -- 172.21.15.160:0/1832630868 wait complete. 2024-05-09T07:40:26.967 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T07:40:27.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.061+0000 7fb5707e7640 1 Processor -- start 2024-05-09T07:40:27.072 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.061+0000 7fb5707e7640 1 -- start start 2024-05-09T07:40:27.073 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb5707e7640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb568152a80 0x7fb5681568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb5707e7640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5680c7ae0 con 0x7fb5680c5690 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb5707e7640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb5680c7dc0 con 0x7fb568155e70 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb5707e7640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb568152e60 con 0x7fb568152a80 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb56e583640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb568152a80 0x7fb5681568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb56ed84640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fb5680c5690 0x7fb56815fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52070/0 (socket says 172.21.15.160:52070) 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.062+0000 7fb56ed84640 1 -- 172.21.15.160:0/4029053530 learned_addr learned my addr 172.21.15.160:0/4029053530 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56e583640 1 -- 172.21.15.160:0/4029053530 >> v1:172.21.15.188:6789/0 conn(0x7fb568155e70 legacy=0x7fb568159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56e583640 1 -- 172.21.15.160:0/4029053530 >> v1:172.21.15.160:6789/0 conn(0x7fb5680c5690 legacy=0x7fb56815fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56e583640 1 -- 172.21.15.160:0/4029053530 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb568158b10 con 0x7fb568152a80 2024-05-09T07:40:27.074 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56d581640 1 -- 172.21.15.160:0/4029053530 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3982709179 0 0) 0x7fb5680c7ae0 con 0x7fb5680c5690 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56e583640 1 --2- 172.21.15.160:0/4029053530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568152a80 0x7fb5681568e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb568152fe0 tx=0x7fb55802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f533d89968d6f686 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56d581640 1 -- 172.21.15.160:0/4029053530 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558037070 con 0x7fb568152a80 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56d581640 1 -- 172.21.15.160:0/4029053530 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb55802ae50 con 0x7fb568152a80 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.063+0000 7fb56d581640 1 -- 172.21.15.160:0/4029053530 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb558032680 con 0x7fb568152a80 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 -- 172.21.15.160:0/4029053530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568152a80 msgr2=0x7fb5681568e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 --2- 172.21.15.160:0/4029053530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568152a80 0x7fb5681568e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb568152fe0 tx=0x7fb55802a100 comp rx=0 tx=0).stop 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 -- 172.21.15.160:0/4029053530 shutdown_connections 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 --2- 172.21.15.160:0/4029053530 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568152a80 0x7fb5681568e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 -- 172.21.15.160:0/4029053530 >> 172.21.15.160:0/4029053530 conn(0x7fb568072ec0 msgr2=0x7fb5680ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 -- 172.21.15.160:0/4029053530 shutdown_connections 2024-05-09T07:40:27.076 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.064+0000 7fb5707e7640 1 -- 172.21.15.160:0/4029053530 wait complete. 2024-05-09T07:40:27.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.065+0000 7fb5707e7640 1 Processor -- start 2024-05-09T07:40:27.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.065+0000 7fb5707e7640 1 -- start start 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb5707e7640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5680c5690 0x7fb568148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb5707e7640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb568152a80 0x7fb568151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb5707e7640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 0x7fb5680aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb5707e7640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb568156320 con 0x7fb568155e70 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb5707e7640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb5680c80a0 con 0x7fb568152a80 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb5707e7640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb5680ad6a0 con 0x7fb5680c5690 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56ed84640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 0x7fb5680aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56ed84640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 0x7fb5680aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34020/0 (socket says 172.21.15.160:34020) 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56ed84640 1 -- 172.21.15.160:0/1308724009 learned_addr learned my addr 172.21.15.160:0/1308724009 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56dd82640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb568152a80 0x7fb568151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:27.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56e583640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5680c5690 0x7fb568148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56ed84640 1 -- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5680c5690 msgr2=0x7fb568148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.066+0000 7fb56ed84640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5680c5690 0x7fb568148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb56ed84640 1 -- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb568152a80 msgr2=0x7fb568151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb56ed84640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb568152a80 0x7fb568151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb56ed84640 1 -- 172.21.15.160:0/1308724009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5680c53a0 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb56ed84640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 0x7fb5680aeb50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c700 tx=0x7fb56000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb560018070 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb56000ce10 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5680b0a30 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb56000a880 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.067+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb568147f60 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.068+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5681eaed0 con 0x7fb568155e70 2024-05-09T07:40:27.081 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.068+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb56001b070 con 0x7fb568155e70 2024-05-09T07:40:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.069+0000 7fb55effd640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb54c06be30 0x7fb54c06e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.069+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6800+0+0 (secure 0 0 0) 0x7fb56008a790 con 0x7fb568155e70 2024-05-09T07:40:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.069+0000 7fb56e583640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb54c06be30 0x7fb54c06e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:27.082 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.069+0000 7fb56e583640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb54c06be30 0x7fb54c06e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb558034000 tx=0x7fb55802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:27.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.073+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb5680b0a30 con 0x7fb568155e70 2024-05-09T07:40:27.305 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.294+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb5680b0c60 con 0x7fb568155e70 2024-05-09T07:40:27.339 INFO:tasks.ceph.mds.d.smithi188.stdout:starting mds.d at 2024-05-09T07:40:27.957 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.946+0000 7fb55effd640 1 -- 172.21.15.160:0/1308724009 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fb5600586b0 con 0x7fb568155e70 2024-05-09T07:40:27.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb54c06be30 msgr2=0x7fb54c06e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb54c06be30 0x7fb54c06e2f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb558034000 tx=0x7fb55802e000 comp rx=0 tx=0).stop 2024-05-09T07:40:27.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 msgr2=0x7fb5680aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:27.960 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 0x7fb5680aeb50 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb56000c700 tx=0x7fb56000cbc0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 shutdown_connections 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb54c06be30 0x7fb54c06e2f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb568152a80 0x7fb568151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb568155e70 0x7fb5680aeb50 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 --2- 172.21.15.160:0/1308724009 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb5680c5690 0x7fb568148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 >> 172.21.15.160:0/1308724009 conn(0x7fb568072ec0 msgr2=0x7fb5681568c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 shutdown_connections 2024-05-09T07:40:27.961 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:27.949+0000 7fb5707e7640 1 -- 172.21.15.160:0/1308724009 wait complete. 2024-05-09T07:40:27.971 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T07:40:28.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.066+0000 7fb449942640 1 Processor -- start 2024-05-09T07:40:28.077 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.066+0000 7fb449942640 1 -- start start 2024-05-09T07:40:28.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb449942640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fb444150a40 0x7fb44406ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:28.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb449942640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb444156890 0x7fb444156d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:28.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb449942640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb44406fe10 con 0x7fb44406f7b0 2024-05-09T07:40:28.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb449942640 1 -- --> v2:172.21.15.160:3300/0 -- mon_getmap magic: 0 -- 0x7fb4440700f0 con 0x7fb444156890 2024-05-09T07:40:28.079 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb449942640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fb44406baa0 con 0x7fb444150a40 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb4437fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb444156890 0x7fb444156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb4437fe640 1 --2- >> v2:172.21.15.160:3300/0 conn(0x7fb444156890 0x7fb444156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34030/0 (socket says 172.21.15.160:34030) 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb4437fe640 1 -- 172.21.15.160:0/2465964943 learned_addr learned my addr 172.21.15.160:0/2465964943 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb442ffd640 1 --2- 172.21.15.160:0/2465964943 >> v2:172.21.15.188:3300/0 conn(0x7fb444150a40 0x7fb44406ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.067+0000 7fb4437fe640 1 -- 172.21.15.160:0/2465964943 >> v2:172.21.15.188:3300/0 conn(0x7fb444150a40 msgr2=0x7fb44406ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb4437fe640 1 --2- 172.21.15.160:0/2465964943 >> v2:172.21.15.188:3300/0 conn(0x7fb444150a40 0x7fb44406ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb4437fe640 1 -- 172.21.15.160:0/2465964943 >> v1:172.21.15.188:6789/0 conn(0x7fb44406f7b0 legacy=0x7fb4440c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb4437fe640 1 -- 172.21.15.160:0/2465964943 --> v2:172.21.15.160:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb44406f130 con 0x7fb444156890 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb442ffd640 1 --2- 172.21.15.160:0/2465964943 >> v2:172.21.15.188:3300/0 conn(0x7fb444150a40 0x7fb44406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb4437fe640 1 --2- 172.21.15.160:0/2465964943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444156890 0x7fb444156d20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb44406b410 tx=0x7fb43c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b22aa49152b406c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb441ffb640 1 -- 172.21.15.160:0/2465964943 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c036070 con 0x7fb444156890 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb441ffb640 1 -- 172.21.15.160:0/2465964943 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb43c031600 con 0x7fb444156890 2024-05-09T07:40:28.080 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.068+0000 7fb441ffb640 1 -- 172.21.15.160:0/2465964943 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43c0319f0 con 0x7fb444156890 2024-05-09T07:40:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 -- 172.21.15.160:0/2465964943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444156890 msgr2=0x7fb444156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 --2- 172.21.15.160:0/2465964943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444156890 0x7fb444156d20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb44406b410 tx=0x7fb43c02a6f0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 -- 172.21.15.160:0/2465964943 shutdown_connections 2024-05-09T07:40:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 --2- 172.21.15.160:0/2465964943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444156890 0x7fb444156d20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 --2- 172.21.15.160:0/2465964943 >> v2:172.21.15.188:3300/0 conn(0x7fb444150a40 0x7fb44406ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.083 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 -- 172.21.15.160:0/2465964943 >> 172.21.15.160:0/2465964943 conn(0x7fb44406c450 msgr2=0x7fb44406c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 -- 172.21.15.160:0/2465964943 shutdown_connections 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.069+0000 7fb449942640 1 -- 172.21.15.160:0/2465964943 wait complete. 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.070+0000 7fb449942640 1 Processor -- start 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.070+0000 7fb449942640 1 -- start start 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb449942640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb44406f7b0 0x7fb4441e97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb449942640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 0x7fb444152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb449942640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb444156890 0x7fb4441dd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb449942640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fb4441573e0 con 0x7fb444150a40 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb449942640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fb444157260 con 0x7fb44406f7b0 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb449942640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fb444157560 con 0x7fb444156890 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 0x7fb444152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 0x7fb444152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34040/0 (socket says 172.21.15.160:34040) 2024-05-09T07:40:28.084 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 -- 172.21.15.160:0/22238879 learned_addr learned my addr 172.21.15.160:0/22238879 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb442ffd640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb44406f7b0 0x7fb4441e97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 -- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb444156890 msgr2=0x7fb4441dd7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb444156890 0x7fb4441dd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 -- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb44406f7b0 msgr2=0x7fb4441e97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb44406f7b0 0x7fb4441e97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.071+0000 7fb4427fc640 1 -- 172.21.15.160:0/22238879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4441dc9f0 con 0x7fb444150a40 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.072+0000 7fb4427fc640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 0x7fb444152f10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb43800b3c0 tx=0x7fb43800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.072+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438011830 con 0x7fb444150a40 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.072+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fb438016070 con 0x7fb444150a40 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.072+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4441dccd0 con 0x7fb444150a40 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.072+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438011c30 con 0x7fb444150a40 2024-05-09T07:40:28.085 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.072+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4441dd290 con 0x7fb444150a40 2024-05-09T07:40:28.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.073+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4441dd490 con 0x7fb444150a40 2024-05-09T07:40:28.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.073+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fb43801b460 con 0x7fb444150a40 2024-05-09T07:40:28.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.074+0000 7fb42bfff640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb43006be30 0x7fb43006e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:28.086 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.074+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6800+0+0 (secure 0 0 0) 0x7fb43808af30 con 0x7fb444150a40 2024-05-09T07:40:28.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.074+0000 7fb442ffd640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb43006be30 0x7fb43006e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:28.087 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.074+0000 7fb442ffd640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb43006be30 0x7fb43006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4440c8040 tx=0x7fb434008040 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:28.092 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.082+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fb4441dd490 con 0x7fb444150a40 2024-05-09T07:40:28.274 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.261+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb4441544c0 con 0x7fb444150a40 2024-05-09T07:40:28.964 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.953+0000 7fb42bfff640 1 -- 172.21.15.160:0/22238879 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7fb4380594b0 con 0x7fb444150a40 2024-05-09T07:40:28.964 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_metadata' removed 2024-05-09T07:40:28.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.962+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb43006be30 msgr2=0x7fb43006e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:28.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.962+0000 7fb449942640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb43006be30 0x7fb43006e2f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb4440c8040 tx=0x7fb434008040 comp rx=0 tx=0).stop 2024-05-09T07:40:28.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.962+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 msgr2=0x7fb444152f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:28.973 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.962+0000 7fb449942640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 0x7fb444152f10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb43800b3c0 tx=0x7fb43800b880 comp rx=0 tx=0).stop 2024-05-09T07:40:28.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.972+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 shutdown_connections 2024-05-09T07:40:28.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.972+0000 7fb449942640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fb43006be30 0x7fb43006e2f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.983 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.972+0000 7fb449942640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fb444156890 0x7fb4441dd7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.972+0000 7fb449942640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb444150a40 0x7fb444152f10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.972+0000 7fb449942640 1 --2- 172.21.15.160:0/22238879 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fb44406f7b0 0x7fb4441e97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:28.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.972+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 >> 172.21.15.160:0/22238879 conn(0x7fb44406c450 msgr2=0x7fb44406c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:28.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.973+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 shutdown_connections 2024-05-09T07:40:28.984 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:28.974+0000 7fb449942640 1 -- 172.21.15.160:0/22238879 wait complete. 2024-05-09T07:40:29.003 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T07:40:29.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.131+0000 7fa5b1a12640 1 Processor -- start 2024-05-09T07:40:29.142 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.131+0000 7fa5b1a12640 1 -- start start 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5b1a12640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7fa5ac152a80 0x7fa5ac1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5b1a12640 1 -- --> v1:172.21.15.160:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ac0c7ae0 con 0x7fa5ac0c5690 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5b1a12640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa5ac0c7dc0 con 0x7fa5ac155e70 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5b1a12640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7fa5ac152e60 con 0x7fa5ac152a80 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5ab7fe640 1 --1- >> v1:172.21.15.160:6789/0 conn(0x7fa5ac0c5690 0x7fa5ac15fd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.160:6789/0 says I am v1:172.21.15.160:52076/0 (socket says 172.21.15.160:52076) 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5ab7fe640 1 -- 172.21.15.160:0/3047820629 learned_addr learned my addr 172.21.15.160:0/3047820629 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:29.146 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/3047820629 >> v2:172.21.15.188:3300/0 conn(0x7fa5ac152a80 0x7fa5ac1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 <== mon.0 v1:172.21.15.160:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 737812738 0 0) 0x7fa5ac0c7ae0 con 0x7fa5ac0c5690 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 --> v1:172.21.15.160:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa598003520 con 0x7fa5ac0c5690 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 <== mon.0 v1:172.21.15.160:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 522625943 0 0) 0x7fa598003520 con 0x7fa5ac0c5690 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 >> v2:172.21.15.188:3300/0 conn(0x7fa5ac152a80 msgr2=0x7fa5ac1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 --2- 172.21.15.160:0/3047820629 >> v2:172.21.15.188:3300/0 conn(0x7fa5ac152a80 0x7fa5ac1568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 >> v1:172.21.15.188:6789/0 conn(0x7fa5ac155e70 legacy=0x7fa5ac159310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 --> v1:172.21.15.160:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac158b10 con 0x7fa5ac0c5690 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 <== mon.0 v1:172.21.15.160:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7fa5a40019f0 con 0x7fa5ac0c5690 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 >> v1:172.21.15.160:6789/0 conn(0x7fa5ac0c5690 legacy=0x7fa5ac15fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5980039b0 0x7fa598003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa598004800 0x7fa598004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980052e0 0x7fa59800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac152e60 con 0x7fa5980039b0 2024-05-09T07:40:29.147 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa59800c1b0 con 0x7fa598004800 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa59800c3b0 con 0x7fa5980052e0 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.132+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5980039b0 0x7fa598003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5a3fff640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa598004800 0x7fa598004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5ab7fe640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980052e0 0x7fa59800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5aaffd640 1 -- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980052e0 msgr2=0x7fa59800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980052e0 0x7fa59800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5aaffd640 1 -- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa598004800 msgr2=0x7fa598004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa598004800 0x7fa598004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5aaffd640 1 -- 172.21.15.160:0/3047820629 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa598003520 con 0x7fa5980039b0 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5ab7fe640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980052e0 0x7fa59800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5a3fff640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa598004800 0x7fa598004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5ab7fe640 1 -- 172.21.15.160:0/3047820629 reap_dead start 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.133+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5980039b0 0x7fa598003df0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ac0 tx=0x7fa598005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164888828ed295bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:29.148 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c036070 con 0x7fa5980039b0 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa59c031400 con 0x7fa5980039b0 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5aa7fc640 1 -- 172.21.15.160:0/3047820629 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa59c009690 con 0x7fa5980039b0 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 -- 172.21.15.160:0/3047820629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5980039b0 msgr2=0x7fa598003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5980039b0 0x7fa598003df0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa59c009ac0 tx=0x7fa598005830 comp rx=0 tx=0).stop 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 -- 172.21.15.160:0/3047820629 shutdown_connections 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/3047820629 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5980039b0 0x7fa598003df0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 -- 172.21.15.160:0/3047820629 >> 172.21.15.160:0/3047820629 conn(0x7fa5ac072ec0 msgr2=0x7fa5ac0ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 -- 172.21.15.160:0/3047820629 shutdown_connections 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.134+0000 7fa5b1a12640 1 -- 172.21.15.160:0/3047820629 wait complete. 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.135+0000 7fa5b1a12640 1 Processor -- start 2024-05-09T07:40:29.149 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.135+0000 7fa5b1a12640 1 -- start start 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5b1a12640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980039b0 0x7fa5ac0abbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5b1a12640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 0x7fa5ac147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5b1a12640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0cb460 0x7fa5ac148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5b1a12640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac156320 con 0x7fa5ac0af840 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5b1a12640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7fa5ac0c80a0 con 0x7fa5ac0cb460 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5b1a12640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7fa5ac0ad6a0 con 0x7fa5980039b0 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 0x7fa5ac147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 0x7fa5ac147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34048/0 (socket says 172.21.15.160:34048) 2024-05-09T07:40:29.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 -- 172.21.15.160:0/1744101636 learned_addr learned my addr 172.21.15.160:0/1744101636 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5ab7fe640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0cb460 0x7fa5ac148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980039b0 0x7fa5ac0abbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 -- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980039b0 msgr2=0x7fa5ac0abbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980039b0 0x7fa5ac0abbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 -- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0cb460 msgr2=0x7fa5ac148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0cb460 0x7fa5ac148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.136+0000 7fa5a3fff640 1 -- 172.21.15.160:0/1744101636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5ac1eea00 con 0x7fa5ac0af840 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.137+0000 7fa5a3fff640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 0x7fa5ac147cd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e600 tx=0x7fa59400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.137+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5940052e0 con 0x7fa5ac0af840 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.137+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fa594018070 con 0x7fa5ac0af840 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.137+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5ac1eece0 con 0x7fa5ac0af840 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.137+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa594003830 con 0x7fa5ac0af840 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.137+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5ac1ef580 con 0x7fa5ac0af840 2024-05-09T07:40:29.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.138+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5ac075650 con 0x7fa5ac0af840 2024-05-09T07:40:29.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.139+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7fa5940039d0 con 0x7fa5ac0af840 2024-05-09T07:40:29.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.139+0000 7fa5a8ff9640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa58c06be60 0x7fa58c06e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:29.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.139+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6362+0+0 (secure 0 0 0) 0x7fa59408afb0 con 0x7fa5ac0af840 2024-05-09T07:40:29.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.139+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa58c06be60 0x7fa58c06e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:29.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.140+0000 7fa5aaffd640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa58c06be60 0x7fa58c06e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa59c033000 tx=0x7fa59c008030 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:29.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.144+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa594059640 con 0x7fa5ac0af840 2024-05-09T07:40:29.333 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.322+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa5ac1567c0 con 0x7fa5ac0af840 2024-05-09T07:40:29.968 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.957+0000 7fa5a8ff9640 1 -- 172.21.15.160:0/1744101636 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7fa594058f40 con 0x7fa5ac0af840 2024-05-09T07:40:29.969 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_data' removed 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa58c06be60 msgr2=0x7fa58c06e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa58c06be60 0x7fa58c06e320 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa59c033000 tx=0x7fa59c008030 comp rx=0 tx=0).stop 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 msgr2=0x7fa5ac147cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 0x7fa5ac147cd0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e600 tx=0x7fa59400eac0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 shutdown_connections 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7fa58c06be60 0x7fa58c06e320 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7fa5ac0cb460 0x7fa5ac148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa5ac0af840 0x7fa5ac147cd0 secure :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa59400e600 tx=0x7fa59400eac0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 --2- 172.21.15.160:0/1744101636 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7fa5980039b0 0x7fa5ac0abbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:29.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.966+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 >> 172.21.15.160:0/1744101636 conn(0x7fa5ac072ec0 msgr2=0x7fa5ac157a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:29.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.967+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 shutdown_connections 2024-05-09T07:40:29.978 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:29.967+0000 7fa5b1a12640 1 -- 172.21.15.160:0/1744101636 wait complete. 2024-05-09T07:40:29.998 DEBUG:teuthology.orchestra.run.smithi160:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-09T07:40:30.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.140+0000 7f513359e640 1 Processor -- start 2024-05-09T07:40:30.151 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.140+0000 7f513359e640 1 -- start start 2024-05-09T07:40:30.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.141+0000 7f513359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 0x7f512c0474a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.141+0000 7f513359e640 1 --2- >> v2:172.21.15.188:3300/0 conn(0x7f512c00a8b0 0x7f512c00ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.141+0000 7f513359e640 1 -- --> v1:172.21.15.188:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f512c00bb50 con 0x7f512c0df180 2024-05-09T07:40:30.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.141+0000 7f513359e640 1 -- --> v2:172.21.15.188:3300/0 -- mon_getmap magic: 0 -- 0x7f512c00a420 con 0x7f512c00a8b0 2024-05-09T07:40:30.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.141+0000 7f513359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f512c048060 con 0x7f512c0dbce0 2024-05-09T07:40:30.152 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.142+0000 7f5131d9b640 1 --1- >> v1:172.21.15.188:6789/0 conn(0x7f512c0df180 0x7f512c006ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.188:6789/0 says I am v1:172.21.15.160:35362/0 (socket says 172.21.15.160:35362) 2024-05-09T07:40:30.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.142+0000 7f5131d9b640 1 -- 172.21.15.160:0/2633299917 learned_addr learned my addr 172.21.15.160:0/2633299917 (peer_addr_for_me v1:172.21.15.160:0/0) 2024-05-09T07:40:30.153 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.142+0000 7f5132d9d640 1 --2- 172.21.15.160:0/2633299917 >> v2:172.21.15.188:3300/0 conn(0x7f512c00a8b0 0x7f512c00ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:30.154 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.144+0000 7f513259c640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 0x7f512c0474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:30.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.144+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 <== mon.1 v1:172.21.15.188:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3947697158 0 0) 0x7f512c00bb50 con 0x7f512c0df180 2024-05-09T07:40:30.155 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.144+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 --> v1:172.21.15.188:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f511c003520 con 0x7f512c0df180 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 <== mon.1 v1:172.21.15.188:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2602368300 0 0) 0x7f511c003520 con 0x7f512c0df180 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 msgr2=0x7f512c0474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 0x7f512c0474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 >> v2:172.21.15.188:3300/0 conn(0x7f512c00a8b0 msgr2=0x7f512c00ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 --2- 172.21.15.160:0/2633299917 >> v2:172.21.15.188:3300/0 conn(0x7f512c00a8b0 0x7f512c00ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 --> v1:172.21.15.188:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c00a0f0 con 0x7f512c0df180 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 <== mon.1 v1:172.21.15.188:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 3535876732 0 0) 0x7f513404fe40 con 0x7f512c0df180 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 >> v1:172.21.15.188:6789/0 conn(0x7f512c0df180 legacy=0x7f512c006ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c0039b0 0x7f511c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.156 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c004350 0x7f511c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 0x7f511c004ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00a420 con 0x7f511c0039b0 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f512c048060 con 0x7f511c004350 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f511c005660 con 0x7f512c0dbce0 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f5132d9d640 1 --2- 172.21.15.160:0/2633299917 >> v2:172.21.15.188:3300/0 conn(0x7f512c00a8b0 0x7f512c00ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f513259c640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c0039b0 0x7f511c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:30.157 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.145+0000 7f5131d9b640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c004350 0x7f511c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.146+0000 7f513259c640 1 -- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 msgr2=0x7f511c004ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.146+0000 7f513259c640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c0dbce0 0x7f511c004ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.146+0000 7f513259c640 1 -- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c004350 msgr2=0x7f511c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.146+0000 7f513259c640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c004350 0x7f511c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.146+0000 7f513259c640 1 -- 172.21.15.160:0/2633299917 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c00a0f0 con 0x7f511c0039b0 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.146+0000 7f5131d9b640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c004350 0x7f511c004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T07:40:30.158 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.147+0000 7f513259c640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c0039b0 0x7f511c003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5128009aa0 tx=0x7f511c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2526f27eab597d79 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:30.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.147+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128036070 con 0x7f511c0039b0 2024-05-09T07:40:30.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.147+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5128031400 con 0x7f511c0039b0 2024-05-09T07:40:30.159 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.147+0000 7f513159a640 1 -- 172.21.15.160:0/2633299917 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5128009670 con 0x7f511c0039b0 2024-05-09T07:40:30.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.149+0000 7f513359e640 1 -- 172.21.15.160:0/2633299917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c0039b0 msgr2=0x7f511c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:30.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.149+0000 7f513359e640 1 --2- 172.21.15.160:0/2633299917 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c0039b0 0x7f511c003dd0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5128009aa0 tx=0x7f511c005830 comp rx=0 tx=0).stop 2024-05-09T07:40:30.160 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.149+0000 7f5132d9d640 1 -- 172.21.15.160:0/2633299917 reap_dead start 2024-05-09T07:40:30.161 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.151+0000 7f513359e640 1 -- 172.21.15.160:0/2633299917 shutdown_connections 2024-05-09T07:40:30.162 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.151+0000 7f513359e640 1 -- 172.21.15.160:0/2633299917 >> 172.21.15.160:0/2633299917 conn(0x7f512c048250 msgr2=0x7f512c048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:30.164 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.154+0000 7f513359e640 1 -- 172.21.15.160:0/2633299917 shutdown_connections 2024-05-09T07:40:30.166 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.154+0000 7f513359e640 1 -- 172.21.15.160:0/2633299917 wait complete. 2024-05-09T07:40:30.167 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.156+0000 7f513359e640 1 Processor -- start 2024-05-09T07:40:30.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.158+0000 7f513359e640 1 -- start start 2024-05-09T07:40:30.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f513359e640 1 --2- >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c0039b0 0x7f512c1781c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f513359e640 1 --2- >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c04a640 0x7f512c174870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.170 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f513359e640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 0x7f512c0428a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f513359e640 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 -- 0x7f512c00b240 con 0x7f512c174db0 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f513359e640 1 -- --> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] -- mon_getmap magic: 0 -- 0x7f512c047d80 con 0x7f511c0039b0 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f513359e640 1 -- --> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] -- mon_getmap magic: 0 -- 0x7f512c0dc0c0 con 0x7f512c04a640 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f5132d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 0x7f512c0428a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.159+0000 7f5132d9d640 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 0x7f512c0428a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:34074/0 (socket says 172.21.15.160:34074) 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.160+0000 7f5132d9d640 1 -- 172.21.15.160:0/1524130588 learned_addr learned my addr 172.21.15.160:0/1524130588 (peer_addr_for_me v2:172.21.15.160:0/0) 2024-05-09T07:40:30.171 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.160+0000 7f5132d9d640 1 -- 172.21.15.160:0/1524130588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c04a640 msgr2=0x7f512c174870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:30.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.160+0000 7f5132d9d640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c04a640 0x7f512c174870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.160+0000 7f5132d9d640 1 -- 172.21.15.160:0/1524130588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c0039b0 msgr2=0x7f512c1781c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:30.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.160+0000 7f5132d9d640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c0039b0 0x7f512c1781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.160+0000 7f5132d9d640 1 -- 172.21.15.160:0/1524130588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f512c0425c0 con 0x7f512c174db0 2024-05-09T07:40:30.172 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.161+0000 7f5132d9d640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 0x7f512c0428a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f512c00beb0 tx=0x7f512400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:30.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.163+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124017070 con 0x7f512c174db0 2024-05-09T07:40:30.173 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.163+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f5124004020 con 0x7f512c174db0 2024-05-09T07:40:30.174 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.163+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134069150 con 0x7f512c174db0 2024-05-09T07:40:30.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.163+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f512c042000 con 0x7f512c174db0 2024-05-09T07:40:30.175 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.163+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f512c174350 con 0x7f512c174db0 2024-05-09T07:40:30.176 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.165+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5118005290 con 0x7f512c174db0 2024-05-09T07:40:30.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.166+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 5) ==== 90498+0+0 (secure 0 0 0) 0x7f512400c8c0 con 0x7f512c174db0 2024-05-09T07:40:30.177 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.167+0000 7f5122ffd640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f511406bf30 0x7f511406e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T07:40:30.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.167+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f512408a940 con 0x7f512c174db0 2024-05-09T07:40:30.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.167+0000 7f513259c640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f511406bf30 0x7f511406e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T07:40:30.178 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.168+0000 7f513259c640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f511406bf30 0x7f511406e3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5128033000 tx=0x7f512802e000 comp rx=0 tx=0).ready entity=mgr.4099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T07:40:30.184 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.173+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5124059240 con 0x7f512c174db0 2024-05-09T07:40:30.372 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.361+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5118005580 con 0x7f512c174db0 2024-05-09T07:40:30.970 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.958+0000 7f5122ffd640 1 -- 172.21.15.160:0/1524130588 <== mon.0 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f5124058b40 con 0x7f512c174db0 2024-05-09T07:40:30.970 INFO:teuthology.orchestra.run.smithi160.stderr:pool 'cephfs_data_ec' removed 2024-05-09T07:40:30.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.961+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f511406bf30 msgr2=0x7f511406e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:30.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.961+0000 7f513359e640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f511406bf30 0x7f511406e3f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5128033000 tx=0x7f512802e000 comp rx=0 tx=0).stop 2024-05-09T07:40:30.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.961+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 msgr2=0x7f512c0428a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T07:40:30.972 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.961+0000 7f513359e640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 0x7f512c0428a0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f512c00beb0 tx=0x7f512400ec90 comp rx=0 tx=0).stop 2024-05-09T07:40:30.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 shutdown_connections 2024-05-09T07:40:30.976 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:6832/2408879908,v1:172.21.15.160:6833/2408879908] conn(0x7f511406bf30 0x7f511406e3f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f512c174db0 0x7f512c0428a0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.188:3301/0,v1:172.21.15.188:6790/0] conn(0x7f512c04a640 0x7f512c174870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 --2- 172.21.15.160:0/1524130588 >> [v2:172.21.15.188:3300/0,v1:172.21.15.188:6789/0] conn(0x7f511c0039b0 0x7f512c1781c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T07:40:30.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 >> 172.21.15.160:0/1524130588 conn(0x7f512c048250 msgr2=0x7f512c048660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T07:40:30.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 shutdown_connections 2024-05-09T07:40:30.977 INFO:teuthology.orchestra.run.smithi160.stderr:2024-05-09T07:40:30.966+0000 7f513359e640 1 -- 172.21.15.160:0/1524130588 wait complete. 2024-05-09T07:40:30.998 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-09T07:40:30.998 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-05-09T07:40:30.999 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:31.012 INFO:tasks.ceph.mds.a:Stopped 2024-05-09T07:40:31.012 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-05-09T07:40:31.012 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:31.067 INFO:tasks.ceph.mds.c:Stopped 2024-05-09T07:40:31.067 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-05-09T07:40:31.067 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:31.078 INFO:tasks.ceph.mds.b:Stopped 2024-05-09T07:40:31.078 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-05-09T07:40:31.078 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:31.128 INFO:tasks.ceph.mds.d:Stopped 2024-05-09T07:40:31.128 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-09T07:40:31.128 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-09T07:40:31.128 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:31.222 INFO:tasks.ceph.osd.0:Stopped 2024-05-09T07:40:31.223 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-09T07:40:31.223 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:35.381 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-05-09T07:40:37.325 INFO:tasks.ceph.osd.1:Stopped 2024-05-09T07:40:37.325 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-09T07:40:37.325 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:37.387 INFO:tasks.ceph.osd.2:Stopped 2024-05-09T07:40:37.388 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-09T07:40:37.388 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:41.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-05-09T07:40:41.290 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-05-09T07:40:43.491 INFO:tasks.ceph.osd.3:Stopped 2024-05-09T07:40:43.492 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-09T07:40:43.492 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.585 INFO:tasks.ceph.osd.4:Stopped 2024-05-09T07:40:43.585 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-09T07:40:43.585 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.676 INFO:tasks.ceph.osd.5:Stopped 2024-05-09T07:40:43.676 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-09T07:40:43.676 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.760 INFO:tasks.ceph.osd.6:Stopped 2024-05-09T07:40:43.760 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-09T07:40:43.760 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.845 INFO:tasks.ceph.osd.7:Stopped 2024-05-09T07:40:43.845 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-09T07:40:43.845 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-09T07:40:43.845 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.873 INFO:tasks.ceph.mgr.y:Stopped 2024-05-09T07:40:43.873 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-09T07:40:43.873 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.905 INFO:tasks.ceph.mgr.x:Stopped 2024-05-09T07:40:43.905 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-09T07:40:43.906 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-09T07:40:43.906 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.927 INFO:tasks.ceph.mon.a:Stopped 2024-05-09T07:40:43.927 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-09T07:40:43.927 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:43.958 INFO:tasks.ceph.mon.b:Stopped 2024-05-09T07:40:43.959 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-09T07:40:43.959 INFO:teuthology.orchestra.run:waiting for 300 2024-05-09T07:40:44.011 INFO:tasks.ceph.mon.c:Stopped 2024-05-09T07:40:44.011 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-09T07:40:44.012 DEBUG:teuthology.orchestra.run.smithi160:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-09T07:40:44.096 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:40:44.096 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-09T07:40:45.127 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:40:45.127 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-09T07:40:45.301 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:40:45.301 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-09T07:40:45.393 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T07:40:45.393 DEBUG:teuthology.orchestra.run.smithi160:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-09T07:40:45.507 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:40:45.507 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-09T07:40:45.652 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:40:45.653 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-09T07:40:45.743 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:40:45.744 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-09T07:40:45.828 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T07:40:45.828 DEBUG:teuthology.orchestra.run.smithi188:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-09T07:40:45.906 INFO:tasks.ceph:Archiving mon data... 2024-05-09T07:40:45.909 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/data/mon.a.tgz 2024-05-09T07:40:45.910 DEBUG:teuthology.orchestra.run.smithi160:> mktemp 2024-05-09T07:40:45.931 INFO:teuthology.orchestra.run.smithi160.stdout:/tmp/tmp.oRizcTQZQE 2024-05-09T07:40:45.931 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.oRizcTQZQE 2024-05-09T07:40:46.097 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 0666 /tmp/tmp.oRizcTQZQE 2024-05-09T07:40:46.200 DEBUG:teuthology.orchestra.remote:smithi160:/tmp/tmp.oRizcTQZQE is 362KB 2024-05-09T07:40:46.291 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-05-09T07:40:46.295 DEBUG:teuthology.orchestra.run.smithi160:> rm -fr /tmp/tmp.oRizcTQZQE 2024-05-09T07:40:46.310 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/data/mon.b.tgz 2024-05-09T07:40:46.311 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-05-09T07:40:46.342 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.3Q8xhm1Ih9 2024-05-09T07:40:46.342 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3Q8xhm1Ih9 2024-05-09T07:40:46.501 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.3Q8xhm1Ih9 2024-05-09T07:40:46.604 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.3Q8xhm1Ih9 is 378KB 2024-05-09T07:40:46.704 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.3Q8xhm1Ih9 2024-05-09T07:40:46.720 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/data/mon.c.tgz 2024-05-09T07:40:46.721 DEBUG:teuthology.orchestra.run.smithi188:> mktemp 2024-05-09T07:40:46.775 INFO:teuthology.orchestra.run.smithi188.stdout:/tmp/tmp.WvOPsUdgK4 2024-05-09T07:40:46.775 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.WvOPsUdgK4 2024-05-09T07:40:46.910 DEBUG:teuthology.orchestra.run.smithi188:> sudo chmod 0666 /tmp/tmp.WvOPsUdgK4 2024-05-09T07:40:46.996 DEBUG:teuthology.orchestra.remote:smithi188:/tmp/tmp.WvOPsUdgK4 is 378KB 2024-05-09T07:40:47.095 DEBUG:teuthology.orchestra.run.smithi188:> rm -fr /tmp/tmp.WvOPsUdgK4 2024-05-09T07:40:47.110 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-09T07:40:47.111 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T07:40:47.113 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-09T07:40:47.199 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T07:40:47.240 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-09T07:40:47.240 INFO:tasks.ceph:Archiving crash dumps... 2024-05-09T07:40:47.242 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/remote/smithi160/crash 2024-05-09T07:40:47.243 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T07:40:47.267 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/remote/smithi188/crash 2024-05-09T07:40:47.268 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-09T07:40:47.304 INFO:tasks.ceph:Shutting down logrotate 2024-05-09T07:40:49.008 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T07:40:49.044 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-05-09T07:40:49.172 INFO:tasks.ceph:Compressing logs... 2024-05-09T07:40:49.172 DEBUG:teuthology.orchestra.run.smithi160:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T07:40:49.175 DEBUG:teuthology.orchestra.run.smithi188:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T07:40:49.204 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35480.log 2024-05-09T07:40:49.204 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T07:40:49.205 INFO:teuthology.orchestra.run.smithi188.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T07:40:49.205 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35480.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35560.log 2024-05-09T07:40:49.205 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35560.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35389.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35640.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35389.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35640.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35600.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42157.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35600.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-09T07:40:49.211 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.1.42157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35520.log 2024-05-09T07:40:49.212 INFO:teuthology.orchestra.run.smithi188.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42157.log.gz 2024-05-09T07:40:49.212 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35431.log 2024-05-09T07:40:49.212 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-05-09T07:40:49.212 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35431.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-05-09T07:40:49.212 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-09T07:40:49.241 INFO:teuthology.orchestra.run.smithi188.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-mon.b.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35480.log.gz 2024-05-09T07:40:49.242 INFO:teuthology.orchestra.run.smithi188.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.35520.log.gz 2024-05-09T07:40:49.242 INFO:teuthology.orchestra.run.smithi188.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35560.log.gz 2024-05-09T07:40:49.242 INFO:teuthology.orchestra.run.smithi188.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35600.log.gz 2024-05-09T07:40:49.242 INFO:teuthology.orchestra.run.smithi188.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35640.log.gz 2024-05-09T07:40:49.247 INFO:teuthology.orchestra.run.smithi188.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-05-09T07:40:49.247 INFO:teuthology.orchestra.run.smithi188.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35389.log.gz 2024-05-09T07:40:49.247 INFO:teuthology.orchestra.run.smithi188.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35431.log.gz 2024-05-09T07:40:49.247 INFO:teuthology.orchestra.run.smithi188.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T07:40:49.257 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45905.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44361.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49547.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49589.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44361.log: /var/log/ceph/ceph-client.admin.49547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35238.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49589.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49547.log.gz 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41474.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.35238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44992.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49589.log.gz 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47379.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44992.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44692.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42524.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48078.log 2024-05-09T07:40:49.258 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45821.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41870.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48582.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48078.log.gz 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42398.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41870.log: /var/log/ceph/ceph-client.admin.48582.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44095.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48582.log.gz 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44355.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.44095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45863.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43597.log 2024-05-09T07:40:49.259 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49337.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45275.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41743.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45653.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49337.log.gz 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45359.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45653.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49086.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43681.log 2024-05-09T07:40:49.260 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49086.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29293.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43681.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49086.log.gz 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42230.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph.tmp-client.admin.29293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48120.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29293.log.gz 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49505.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44685.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48120.log.gz 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.49505.log: /var/log/ceph/ceph-client.admin.47547.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45569.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49505.log.gz 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47547.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44350.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48708.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44996.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44350.log: gzip/var/log/ceph/ceph-client.admin.48708.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35605.log 2024-05-09T07:40:49.261 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43245.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48708.log.gz 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.35605.log: /var/log/ceph/ceph-client.admin.43245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45443.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45905.log.gz 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44997.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42482.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44698.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48750.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44698.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45821.log.gz 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5/var/log/ceph/ceph-client.admin.48750.log: --verbose -- /var/log/ceph/ceph-client.admin.44683.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mds.c.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.48750.log.gz 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44053.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47195.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43471.log 2024-05-09T07:40:49.262 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44053.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45863.log.gz 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47195.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45653.log.gz 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43801.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42084.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.43801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44137.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44684.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:gzip/var/log/ceph/ceph-client.admin.44137.log: -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48288.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44694.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-osd.0.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35238.log.gz 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46067.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48288.log: /var/log/ceph/ceph-client.admin.44694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45059.log 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 2024-05-09T07:40:49.263 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.46067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49002.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41743.log.gz 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:gzip/var/log/ceph/ceph-client.admin.45059.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44697.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41870.log.gz 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45779.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44697.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49002.log.gz 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42590.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43413.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43759.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48162.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42272.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41785.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42272.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48162.log.gz 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48624.log 2024-05-09T07:40:49.264 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43927.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr: 88.6% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35605.log.gz -- replaced with /var/log/ceph/ceph-client.admin.48624.log.gz 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45611.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43029.log: /var/log/ceph/ceph-client.admin.43927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45737.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47279.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45737.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49044.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47321.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48540.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47321.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49044.log.gz 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35502.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49463.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.35502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47505.log 2024-05-09T07:40:49.265 INFO:teuthology.orchestra.run.smithi160.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48540.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48792.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.49463.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43555.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48960.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44095.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43555.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48792.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43513.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44355.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46115.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48960.log: gzip/var/log/ceph/ceph-client.admin.43513.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49631.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44361.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44350.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.48960.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.46115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48204.log 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44692.log.gz 2024-05-09T07:40:49.266 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49631.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44137.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45233.log 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.49631.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:gzip/var/log/ceph/ceph-client.admin.48204.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44697.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41996.log 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44694.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.48204.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44683.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47237.log 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44685.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42440.log 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48372.log 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44698.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44684.log.gz 2024-05-09T07:40:49.267 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49421.log 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48414.log 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44992.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44997.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.48372.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48876.log 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44996.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49421.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45527.log 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45059.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48414.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49255.log 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45275.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45359.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48876.log.gz 2024-05-09T07:40:49.268 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44897.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41662.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44897.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.49255.log.gz 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43329.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49379.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44358.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49170.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46508.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.49379.log.gz 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.49170.log.gz 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.0.46508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44990.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42186.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45947.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48330.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr: 87.5%/var/log/ceph/ceph-client.admin.45947.log: -- replaced with /var/log/ceph/ceph-client.admin.42230.log.gz 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-05-09T07:40:49.269 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43969.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42398.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48330.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42144.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42482.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42314.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45019.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43969.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43597.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43681.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48498.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48918.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45019.log: /var/log/ceph/ceph-client.admin.48498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43203.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48918.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48498.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44359.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48918.log.gz 2024-05-09T07:40:49.270 INFO:teuthology.orchestra.run.smithi160.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45737.log.gz 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45779.log.gz 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48666.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47421.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43287.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47421.log: gzip -5 --verbose -- 89.2% /var/log/ceph/ceph-client.admin.48834.log -- replaced with /var/log/ceph/ceph-client.admin.48666.log.gz 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48834.log.gz 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44682.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47153.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49213.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47153.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48246.log 2024-05-09T07:40:49.271 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44360.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48246.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49213.log.gz 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48246.log.gz 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47111.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43639.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43885.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41566.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45695.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48456.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41828.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.48456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43843.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41828.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48456.log.gz 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49295.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42040.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43371.log 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.49295.log.gz 2024-05-09T07:40:49.272 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45317.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.43371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42356.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.45317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47463.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.42356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41954.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47589.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.41954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49128.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.47589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44011.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.49128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44179.log 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44011.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.49128.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr:/var/log/ceph/ceph-client.admin.44179.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43245.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.42524.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43513.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43413.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43471.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43555.log.gz 2024-05-09T07:40:49.273 INFO:teuthology.orchestra.run.smithi160.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41662.log.gz 2024-05-09T07:40:49.274 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42590.log.gz 2024-05-09T07:40:49.274 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41785.log.gz 2024-05-09T07:40:49.274 INFO:teuthology.orchestra.run.smithi160.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41828.log.gz 2024-05-09T07:40:49.275 INFO:teuthology.orchestra.run.smithi160.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47111.log.gz 2024-05-09T07:40:49.275 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47153.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47195.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47321.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47279.log.gz -- replaced with /var/log/ceph/ceph-client.admin.47237.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47379.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47463.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47505.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47421.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47589.log.gz 2024-05-09T07:40:49.276 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45019.log.gz 2024-05-09T07:40:49.282 INFO:teuthology.orchestra.run.smithi160.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45233.log.gz 2024-05-09T07:40:49.282 INFO:teuthology.orchestra.run.smithi160.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45317.log.gz 2024-05-09T07:40:49.291 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45695.log.gz 2024-05-09T07:40:49.301 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46067.log.gz 2024-05-09T07:40:49.301 INFO:teuthology.orchestra.run.smithi160.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46115.log.gz 2024-05-09T07:40:49.301 INFO:teuthology.orchestra.run.smithi160.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.45947.log.gz 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: 88.1% 89.3% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43843.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43969.log.gz 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: 87.9% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43801.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43885.log.gz 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: 87.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44011.log.gz -- replaced with /var/log/ceph/ceph-client.admin.43927.log.gz 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.303 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43759.log.gz 2024-05-09T07:40:49.304 INFO:teuthology.orchestra.run.smithi160.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43203.log.gz 2024-05-09T07:40:49.305 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43329.log.gz 2024-05-09T07:40:49.306 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43371.log.gz 2024-05-09T07:40:49.306 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43287.log.gz 2024-05-09T07:40:49.307 INFO:teuthology.orchestra.run.smithi160.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.44053.log.gz 2024-05-09T07:40:49.308 INFO:teuthology.orchestra.run.smithi160.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41566.log.gz 2024-05-09T07:40:49.315 INFO:teuthology.orchestra.run.smithi160.stderr: 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-09T07:40:49.324 INFO:teuthology.orchestra.run.smithi160.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.42314.log.gz 2024-05-09T07:40:49.324 INFO:teuthology.orchestra.run.smithi160.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42272.log.gz 2024-05-09T07:40:49.325 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42356.log.gz 2024-05-09T07:40:49.325 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42440.log.gz 2024-05-09T07:40:49.325 INFO:teuthology.orchestra.run.smithi160.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43639.log.gz 2024-05-09T07:40:49.330 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41474.log.gz 2024-05-09T07:40:49.332 INFO:teuthology.orchestra.run.smithi160.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35502.log.gz 2024-05-09T07:40:49.347 INFO:teuthology.orchestra.run.smithi160.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-05-09T07:40:49.354 INFO:teuthology.orchestra.run.smithi160.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41954.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41996.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42084.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42186.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45443.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45611.log.gz 2024-05-09T07:40:49.358 INFO:teuthology.orchestra.run.smithi160.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42040.log.gz 2024-05-09T07:40:49.359 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45569.log.gz 2024-05-09T07:40:49.359 INFO:teuthology.orchestra.run.smithi160.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45527.log.gz 2024-05-09T07:40:49.378 INFO:teuthology.orchestra.run.smithi160.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-05-09T07:40:49.378 INFO:teuthology.orchestra.run.smithi188.stderr: 92.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-09T07:40:49.397 INFO:teuthology.orchestra.run.smithi160.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44179.log.gz 2024-05-09T07:40:49.397 INFO:teuthology.orchestra.run.smithi160.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44359.log.gz 2024-05-09T07:40:49.397 INFO:teuthology.orchestra.run.smithi160.stderr: 88.5% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44360.log.gz -- replaced with /var/log/ceph/ceph-client.admin.44358.log.gz 2024-05-09T07:40:49.398 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:40:49.398 INFO:teuthology.orchestra.run.smithi160.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2024-05-09T07:40:49.398 INFO:teuthology.orchestra.run.smithi160.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-05-09T07:40:49.398 INFO:teuthology.orchestra.run.smithi160.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44682.log.gz 2024-05-09T07:40:49.398 INFO:teuthology.orchestra.run.smithi160.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44897.log.gz 2024-05-09T07:40:49.398 INFO:teuthology.orchestra.run.smithi160.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-05-09T07:40:49.399 INFO:teuthology.orchestra.run.smithi160.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44990.log.gz 2024-05-09T07:40:49.419 INFO:teuthology.orchestra.run.smithi188.stderr: 94.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-09T07:40:49.486 INFO:teuthology.orchestra.run.smithi188.stderr: 94.2% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-05-09T07:40:51.131 INFO:teuthology.orchestra.run.smithi160.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-09T07:40:55.962 INFO:teuthology.orchestra.run.smithi160.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-09T07:40:57.130 INFO:teuthology.orchestra.run.smithi188.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-09T07:40:57.221 INFO:teuthology.orchestra.run.smithi160.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-09T07:41:41.562 INFO:teuthology.orchestra.run.smithi188.stderr: 94.0% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-09T07:42:45.155 INFO:teuthology.orchestra.run.smithi160.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.0.46508.log.gz 2024-05-09T07:43:05.218 INFO:teuthology.orchestra.run.smithi188.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-05-09T07:45:27.151 INFO:teuthology.orchestra.run.smithi160.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-09T07:46:02.840 INFO:teuthology.orchestra.run.smithi160.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-09T07:46:22.257 INFO:teuthology.orchestra.run.smithi160.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-09T07:46:22.259 INFO:teuthology.orchestra.run.smithi160.stderr: 2024-05-09T07:46:22.259 INFO:teuthology.orchestra.run.smithi160.stderr:real 5m33.070s 2024-05-09T07:46:22.259 INFO:teuthology.orchestra.run.smithi160.stderr:user 4m7.530s 2024-05-09T07:46:22.259 INFO:teuthology.orchestra.run.smithi160.stderr:sys 0m10.203s 2024-05-09T07:54:28.206 INFO:teuthology.orchestra.run.smithi188.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-09T07:55:25.717 INFO:teuthology.orchestra.run.smithi188.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-09T07:56:13.319 INFO:teuthology.orchestra.run.smithi188.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-09T07:56:27.331 INFO:teuthology.orchestra.run.smithi188.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-09T07:56:27.332 INFO:teuthology.orchestra.run.smithi188.stderr: 2024-05-09T07:56:27.332 INFO:teuthology.orchestra.run.smithi188.stderr:real 15m38.136s 2024-05-09T07:56:27.333 INFO:teuthology.orchestra.run.smithi188.stderr:user 7m41.864s 2024-05-09T07:56:27.333 INFO:teuthology.orchestra.run.smithi188.stderr:sys 0m20.375s 2024-05-09T07:56:27.333 INFO:tasks.ceph:Archiving logs... 2024-05-09T07:56:27.335 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/remote/smithi160/log 2024-05-09T07:56:27.336 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T07:57:29.064 DEBUG:teuthology.misc:Transferring archived files from smithi188:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/remote/smithi188/log 2024-05-09T07:57:29.065 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T08:00:00.361 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T08:00:00.370 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T08:00:00.370 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T08:00:00.374 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T08:00:00.547 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T08:00:00.560 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T08:00:00.560 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:00:00.563 DEBUG:teuthology.orchestra.run.smithi188:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T08:00:00.578 INFO:teuthology.orchestra.run.smithi160.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:00:00.580 INFO:teuthology.orchestra.run.smithi188.stderr:bash: line 1: ntpq: command not found 2024-05-09T08:00:00.608 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:00:00.608 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2024-05-09T08:00:00.608 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv01.front.sepia.ceph.com 3 6 377 19 +820us[ +976us] +/- 57ms 2024-05-09T08:00:00.608 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 16 -657us[ -657us] +/- 50ms 2024-05-09T08:00:00.608 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -462us[ -307us] +/- 83ms 2024-05-09T08:00:00.609 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:00:00.621 INFO:teuthology.orchestra.run.smithi188.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T08:00:00.621 INFO:teuthology.orchestra.run.smithi188.stdout:=============================================================================== 2024-05-09T08:00:00.621 INFO:teuthology.orchestra.run.smithi188.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +861us[ +825us] +/- 57ms 2024-05-09T08:00:00.621 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 -586us[ -623us] +/- 50ms 2024-05-09T08:00:00.621 INFO:teuthology.orchestra.run.smithi188.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 -407us[ -444us] +/- 83ms 2024-05-09T08:00:00.621 INFO:teuthology.orchestra.run.smithi188.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T08:00:00.622 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T08:00:00.632 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T08:00:00.633 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T08:00:00.641 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T08:00:00.778 DEBUG:teuthology.orchestra.run.smithi188:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T08:00:00.859 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:00:00.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:00:00.890 DEBUG:teuthology.orchestra.run.smithi188:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T08:00:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:00:00.920 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T08:00:00.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T08:00:00.942 INFO:teuthology.task.internal:Duration was 3183.828094 seconds 2024-05-09T08:00:00.943 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T08:00:00.952 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T08:00:00.952 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T08:00:00.955 DEBUG:teuthology.orchestra.run.smithi188:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T08:00:01.010 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:00:01.020 INFO:teuthology.orchestra.run.smithi188.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T08:00:01.488 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T08:00:01.489 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2024-05-09T08:00:01.489 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T08:00:01.510 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi188.front.sepia.ceph.com 2024-05-09T08:00:01.510 DEBUG:teuthology.orchestra.run.smithi188:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T08:00:01.544 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T08:00:01.544 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T08:00:01.553 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T08:00:01.614 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T08:00:01.614 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T08:00:01.620 DEBUG:teuthology.orchestra.run.smithi188:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T08:00:01.768 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T08:00:01.778 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T08:00:01.778 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T08:00:01.808 DEBUG:teuthology.orchestra.run.smithi188:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T08:00:01.848 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T08:00:01.859 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:00:01.861 DEBUG:teuthology.orchestra.run.smithi188:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:00:01.893 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2024-05-09T08:00:01.926 INFO:teuthology.orchestra.run.smithi188.stdout:kernel.core_pattern = core 2024-05-09T08:00:01.949 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:00:01.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:00:01.972 DEBUG:teuthology.orchestra.run.smithi188:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T08:00:02.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T08:00:02.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T08:00:02.014 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T08:00:02.015 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/remote/smithi160 2024-05-09T08:00:02.015 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T08:00:02.051 DEBUG:teuthology.misc:Transferring archived files from smithi188:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-08_21:24:04-fs-squid-distro-default-smithi/7698588/remote/smithi188 2024-05-09T08:00:02.051 DEBUG:teuthology.orchestra.run.smithi188:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T08:00:02.085 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T08:00:02.085 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T08:00:02.087 DEBUG:teuthology.orchestra.run.smithi188:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T08:00:02.134 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T08:00:02.145 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T08:00:02.146 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T08:00:02.156 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T08:00:02.156 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T08:00:02.158 DEBUG:teuthology.orchestra.run.smithi188:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T08:00:02.171 INFO:teuthology.orchestra.run.smithi160.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 08:00 /home/ubuntu/cephtest 2024-05-09T08:00:02.188 INFO:teuthology.orchestra.run.smithi188.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 08:00 /home/ubuntu/cephtest 2024-05-09T08:00:02.189 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T08:00:02.202 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T08:00:02.225 INFO:teuthology.run:Summary data: description: fs/mixed-clients/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-2c-client conf/{client mds mgr mon osd} distro/{centos_latest} kclient-overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped} objectstore-ec/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health} tasks/kernel_cfuse_workunits_dbench_iozone} duration: 3183.828094482422 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-09T08:00:02.225 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T08:00:02.314 INFO:teuthology.run:pass